Sunil S. Ranka's Weblog

Superior Data Analytics is the antidote to Business Failure

Archive for the ‘sunil s ranka’ Category

Behind The Scene Of MapReduce Job

Posted by sranka on October 28, 2013

Hi All

Recently I have spending most of my time on Big Data projects,using CDH 4.X. Understanding key component of hadoop infrastruture  is very necessary, But the MapReduce (MR) is the most important for processing and aggregrating data. For getting the best of the performance, one needs to know the details of MapReduce job. After reading several white papers and few books, in my opinion below paragraph summarizes the MapReduce THE BEST !!!!!

All About Map Reduce 

The execution of a MapReduce job is broken down into map tasks and reduce tasks. Subsequently, map task execution is divided into the phases: Read (reading map inputs), Map (map function pro-cessing), Collect (serializing to buffer and partitioning), Spill (sorting, combining, compressing, and writing map outputs to local disk), and Merge (merging sorted spill files). Reduce task execution is divided into the phases: Shuffle (transferring map outputs to reduce tasks, with decompression if needed), Merge (merging sorted map outputs), Reduce (reduce function processing), and Write (writing reduce outputs to the distributed file-system). Each phase represents an important part of
the job’s overall execution in Hadoop.

In the MapReduce model, computation is divided into a map function and a reduce function. The map function takes a key/value pair and produces one or more intermediate key/value pairs. The reduce function then takes these intermediate key/value pairs and merges all values corresponding to a single key. The map function can run independently on each key/value pair, exposing enormous amounts of parallelism. Similarly, the reduce function can run independently on each intermediate key, also exposing significant parallelism. In Hadoop, a centralized JobTracker service is responsible for splitting the input data into pieces for processing by independent map and reduce tasks, scheduling each task on a cluster node for execution, and recovering from failures by re-running tasks. On each node, a TaskTracker service runs MapReduce tasks and periodically contacts the JobTracker to report task completions and request new tasks. By default, when a new task is received, a new JVM instance will be spawned to execute it.

The about text is taken from

The Hadoop Distributed Filesystem: Balancing Portability and Performance by Jeffrey Shafer, Scott Rixner, and Alan L. Cox :Rice University

Technical Report : Hadoop Performance Models By Herodotos Herodotou 

 

Hope This helps

Sunil S Ranka

“Superior BI is the antidote to Business Failure”

Posted in Big Data, sunil s ranka | Tagged: , , , , , , , , | Leave a Comment »

OBIEE – 11.1.1.6 At a Glance

Posted by sranka on February 23, 2012

Hi All

Finally cat is out of the bag, the much awaited and anticipated OBIEE 11.1.1.6 is out and available for GA.

The new release adds more enhancements, additional certifications, and bug fixes among other features. This release delivers a BI platform specially designed to leverage:

  • Oracle Exalytics hardware’s large memory, processors, concurrency, and other hardware features and system configurations
  • Enhancements to Times-Ten for Exalytics for analytical processing at in-memory speeds
  • Dynamic user interface enhancements that complement large amounts of data to present business information in meaningful and compelling ways
  • A new BI Server Summary Advisor for Exalytics for aggregate generation and persistence
  • Essbase memory usage optimizations and concurrency improvements for Exalytics to deliver efficient distribution of processing
  • BI Publisher performance, lifecycle, workflow and report creation enhancements
  • New and enhanced Scorecard views and BI Mobile improvements
  • Numerous Security, Management/Diagnostic and Lifecycle enhancements
  • Certified BI and EPM Applications on Exalytics

Hope This helps

Sunil S Ranka

“Superior BI is the antidote to Business Failure”

Posted in 11g, sunil s ranka | Tagged: , , , , | Leave a Comment »

People who all influenced me in Year 2011

Posted by sranka on December 29, 2011

Dear All,

I was sitting around and thinking about few people , whom I met during the course of 2011, some of them left a mark and impression, hence thought of mentioning them in words.

Personally it was satisfactory year,

  • The project I implemented,won “Fusion Middleware Innovation Awards”
  • Implemented 11G with BW-SAP as datasource (odd, on which oracle also wouldn’t bet)
Oracle Fusion Middleware Innovation Award

Oracle Fusion Middleware Innovation Award

  • Met some of the people whom I had always admired likes of murali subramanian and Balaji Yelamanchili from Oracle, Ming Tsai of infosys to mention few.
Murali Subramanian, GM Oracle , India

Murali Subramanian Larry Ellison Of Oracle, India

During my day at Oracle, Murali was talk of town, he was first one to venture into India and grew Oracle’s IDC (India Development Center) from no where to little over 10,000 people in couple of years. I would attribute most of Oracle’s outsourcing success to Murali. 2 hours I spent with him were one of the most cherished moment of OOW and 2011. A truly amazing and down to earth personality.

Balaji Yelamanchili

Balaji Yelamanchili From Oracle

I had opportunity to meet Balaji during collaborate-11 at florida, on behalf of BIWA board, I had invited him as a featured guest. His candid personality was simply astonishing, in spite of being at the that level his eagerness to learn BI was commendable. His customer focused attitude helped one of our key customer to go live against all odds. I heard him again at OOW-11, his transformation from EMC, Documentum to Oracle BI is simply amazing. In all, Oracle BI is in safe hands.

Ming Tsai From Infosys, Managing Director & Chief Client Officer

Ming Tsai From Infosys, Managing Director & Chief Client Officer

I had heard Ming on couple of occasions (OOW-10 and OOW-11), his unique style of presentation had made an impact, even though during OOW-10, kris gopalakrishnan’s presentation was dry, Ming made it interesting, he did justice once again to S.D. Shibulal’s OOW-11 presentation. His stage presence and charisma is some thing adorable. I run into Ming Tsai, during OOW’s Treasure Island party. He was nice and kind enough to spend some time,talk and most importantly he helped setting up camera for nice shot. In short he is well rounded .

Met few more smart people , The Real OBIEE 11G Heros, please read post https://sranka.wordpress.com/2011/06/23/453/

Thanks You All for reading following.

Hoping and wishing you all a happy and prosperous new year. Be safe and enjoy.

Sunil S Ranka

“Superior BI is the antidote to Business Failure”

Posted in Me, sunil s ranka | Tagged: , , , , , , , , , | Leave a Comment »

Default OBIEE 11G Ports

Posted by sranka on December 28, 2011

Hi All

Recently came across a situation, in which someone  Due to some security issue someone wanted to bypass OBIEE’s default ports. After looking at multiple places, I could put all information together. I thought it would be a great share.

Even out of the box installation lets you change the configuration, during installation.

Once we selected one installation type from either Enterprise Install or Software only install and click next. Then in the next step it will ask for port configuration. Check Specify Ports using Configuration file and browse Just specify the newly assign port configuration file (staticports.ini , see the format below).

Default Port Numbers :

WEBLOGIC

  • Domain Port No = 7001
  • Oracle WLS BIEE Managed Server Port No = 9704
  • Oracle WLS BIEE Managed Server SSL Port No = 9804

OPMN

  • Oracle Process Manager Local Port No = 6100
  • Oracle Process Manager Remote Port No = 6200
  • Oracle Process Manager Request Port No = 6003

BIFOUNDATION

  • Oracle BI Server Port No = 9703
  • Oracle BI Server Monitor Port No = 9701
  • Oracle BI Presentation Services Port No = 9710
  • Oracle BI Scheduler Port No = 9705
  • Oracle BI Scheduler Monitor Port No = 9708
  • Oracle BI Scheduler Script RPC Port No = 9707
  • Oracle BI ClusterController Port No = 9706
  • Oracle BI ClusterController Monitor Port No = 9700
  • Oracle BI JavaHost Port No = 9810

Format of staticports.ini File


[WEBLOGIC]

#The Domain port no. This is the listen port of Weblogic Adminserver for the domain.
Domain Port No = 7001

#The "content" port for the BIEE apps. This is the Weblogic Managed Server port on which BIEE applications are deployed.
Oracle WLS BIEE Managed Server Port No = 9704

#The SSL port for the Weblogic Managed Server
Oracle WLS BIEE Managed Server SSL Port No = 9804

[OPMN]

#Process Manager Local port no
Oracle Process Manager Local Port No = 6700

#Process Manager Remote port no
Oracle Process Manager Remote Port No = 6701

#Process Manager Request port no
Oracle Process Manager Request Port No = 6702

[BIFOUNDATION]

#The listen port for OracleBIServer component
Oracle BI Server Port No = 9703

#The monitor port for OracleBIServer component
Oracle BI Server Monitor Port No = 9701

#The listen port for OracleBIPresentationServices component
Oracle BI Presentation Services Port No = 9710

#The listen port for OracleBIScheduler component
Oracle BI Scheduler Port No = 9705

#The monitor port for OracleBIScheduler component
Oracle BI Scheduler Monitor Port No = 9708

#The script RPC port for OracleBIScheduler component
Oracle BI Scheduler Script RPC Port No = 9707

#The listen port for OracleBIClusterController component
Oracle BI ClusterController Port No = 9706

#The monitor port for OracleBIClusterController component
Oracle BI ClusterController Monitor Port No = 9700

#The listen port for OracleBIJavaHost component
Oracle BI JavaHost Port No = 9810

Hope This helps.

Sunil S Ranka

“Superior BI is the antidote to Business Failure”

Posted in 11g, OBIEE, sunil s ranka | Tagged: , , , , , | 2 Comments »

OBIEE 11g : File is too large, error while uploading RPD more than 40MB

Posted by sranka on October 26, 2011

Hi All

Recently ran into an very interesting problem, where using EM when we trying uploading RPD of 40MB size, we repeatedly failed with UI Error message:”Warning: The file upload failed. The file could not be uploaded because it is too large..” .  Sai Kodthivada, from team, did some reasearch and found that there is harcoding of 40MB in File location:

D:\OBIEE11g\user_projects\domains\bifoundation_domain\servers\AdminServer\tmp\_WL_user\em\jmb4hf\war\WEB-INF\web.xml
<!-- Maximum memory per request (in bytes) -->
<context-param>
   <param-name>org.apache.myfaces.trinidad.UPLOAD_MAX_MEMORY</param-name>
<param-value>1024000</param-value>
</context-param>

<!-- Maximum disk space per request (in bytes) -->
<context-param>
<param-name>org.apache.myfaces.trinidad.UPLOAD_MAX_DISK_SPACE</param-name>
<param-value>50960000</param-value>
</context-param>

We tried modifying the file, but we were not successful, after filing an SR with oracle, support came with an alternative.
An alternative to using EM to upload an RPD may be found in Doc ID: “How To Deploy An OBIEE 11g RPD Using An MBean Via Enterprise Manager And A WLST Script (Doc ID 1319333.1)”.

Hope This helps.

Sunil S Ranka

“Superior BI is the antidote to Business Failure”

Hope This helps.

Sunil S Ranka

“Superior BI is the antidote to Business Failure”

Posted in OBIEE, sunil s ranka | Tagged: , , , , , , | Leave a Comment »

Sample OBIEE 11g Usage Tracking Repository

Posted by sranka on September 27, 2011

Hi All

Some of you must have ran into the issue where OBIEE 11g Usage Tracking RPD Fails To Open In Administrator With [nQSError: 36015]. After working with oracle they helped us with sample usge tracking repository . Thanks to Steve Fitzgerald from Oracle , in anticipation of customers needing the file, he attached the RPDs to the following doc.

The Sample OBIEE 11g Usage Tracking RPD Fails To Open In Administrator With [nQSError: 36015] (Doc ID 1336567.1)

Hope this helps

Sunil S Ranka
“Superior BI is the antidote to Business Failure”

Posted in 11g, OBIEE, sunil s ranka | Tagged: , , , , , , , | 1 Comment »

Unable To Create Session Variable Using Obiee 11.1.1.5 Admin Tool

Posted by sranka on June 2, 2011

Hi All

An interesting thing happen, when I was trying to choose a connection pool for creating a Session variable Init Block , connection pool Select button was disabled. This functionality worked in 11.1.1.3. Just to avoid any doubt I restarted admin tool, followed by machine restart, but end result didn’t change. After looking around, I remembered that in one of the metalink they talked about “Allow First Connection for init block”. I tried that option and it work . Below are the details ::

Issue :

Select button is disabled when trying to choose a connection pool for creating a Session variable Init Block

Solution:

Check “Allow First Connection for init block” to true

For setting “Allow First Connection for init block”  follow below steps :

  1. In the Administration Tool, select Tools, then select Options to display the Options dialog.
  2. On the General tab, select “Allow First Connection for init block. (this option is not a best practice and might cause performance issues.)

Hope This Helps

Sunil S Ranka
“Superior BI is the antidote to Business Failure”

 

Posted in OBIEE, sunil s ranka | Tagged: , , , , | 4 Comments »

Fight of co-existance :: Application Role (OBIEE 11g) and Groups (OBIEE 10g)

Posted by sranka on April 22, 2011

Hi All

Past few I have been implementing OBIEE 11g with BW SAP backend/data source. The journey wasn’t easy and challenges are still not over. But during 10g upgrade, I came across an interesting situation where I had to design security for 11g OBIEE but support 10g security from upgrade in the same rpd and webcatalogue

As we all are aware 10g requires GROUPS (data/object level security) and WEBGROUPS (Webcatalogue level object security) for security. Due to backward compatibility GROUPS and WEBGROUPS are present in 11g as well.

Interesting Observation :

When you set ROLES and GROUPS session variable, ROLES takes precedence over GROUPS and only ROLES values are present, when you click on My Account –> Roles and Group. This could be potential oracle bug, but in my case we reverted back to 10g security where we are continuing to use GROUPS and WEBGROUPS variable.

Hope This Helps

Sunil S Ranka
“Superior BI is the antidote to Business Failure”

Posted in 11g, OBIEE, sunil s ranka | Tagged: , , , , | 1 Comment »

How does SAW cache work ?

Posted by sranka on September 25, 2010

Hi All

Came across an interesting problem recently. The issue was when reports were generating the same logical SQL, inspite of NO cache setting the results were cached. All the setting  CacheMinUserExpireMinutes, CacheMinExpireMinutes,  CacheMaxEntries in instanceConfig.xml were set to 0 (zero).

After debugging the issue it was nail down to an interesting fact that , if the same operation is done within an intervel of 30-38 sec, the results were getting refreshed. Following note from metalink made the finding more concrete.

Details on How does SAW Cache Work

The Analytics Engine uses the LRU algorithm (Least recently used) to clean out the cache entries. This works in conjunction with the parameter settings.

Anything that exists longer than CacheMaxExpireMinutes may and probably will be expired once the cleanup routine (which runs every minute or so) detects it. This is regardless of the CacheMinExpireMinutes setting.

When CacheMaxEntries is exceeded it will try to get rid of anything any entries with a time greater than CacheMinExpireMinutes (in LRU order though). In extreme cases it won’t even honour the CacheMinExpireMinutes setting.

The CacheMinUserExpireMinutes will generally keep a cache entry around for that # of minutes longer (even if CacheMinExpireMins or CacheMaxExpireMins have been exceeded). It’s user specific though in that other users will not use a cache entry that is being kept alive by another user if CacheMaxExpireMins has been exceeded.

Hope This Help

Sunil S Ranka

“Superior BI is the antidote to Business Failure”

Posted in OBIEE, sunil s ranka | Tagged: , , , | 1 Comment »

The Myth : Usage Tracking Measures For Tracking The Performance Of Report Retrieval Time

Posted by sranka on June 22, 2010

Hi All,

Took some time off from the blog and other thing to take care of some family priorities, and now back to reality of OBIEE world.  I had been on a Ops call for more than 7 days, issue was very typical, during this debug I explored Usage Tracking in detail.  Following are the detail, this was taken from Doc ID 973090.1]

TOTAL_TIME_SEC is not a good measure of the time it takes from user submittal to retrieval completion because OBI has a zero footprint client, and Usage Tracking is only able to account for the time from when a request enters the OBI Presentation Server to the time that OBIPS releases the results to the client. The Time spent between UI and OBIPS is not accounted for in Usage Tracking.TOTAL_TIME_SEC is the time that the OBI server spent working on a query.This includes the clock time spent waiting for queries to complete. If multiple physical queries are running in parallel, the time reported is how long OBI is spent waiting for the database.This basically is the run time of the longest running query. The total time of all physical queries spawned by a single logical query is reported in CUM_DB_TIME_SEC. This is not part of TOTAL_TIME_SEC. Time spent waiting for resources is not included in TOTAL_TIME_SEC.Note also that START_TS and END_TS have nothing at all to do with TOTAL_TIME_SEC.

START_TS is the time when a user submits a query.This could be when they hit the Results tab in Answers or when they select a dashboard page.
END_TS is when the results are returned to the client.The difference between START_TS and END_TS also includes any time spent waiting for resources, such as waiting for a free connection. In the example where the TOTAL_TIME_SEC is 0 but the difference between START_TS and END_TS is 14 minutes, the 14 minutes could have been spent waiting.

If it is evident that a report is delayed, it is because it is waiting for a connection pool connection, a dbgateway thread or a server thread.
Connection pool connections are set in the .rpd. Dbgateway and server threads are set in nqsconfig.ini. Connection pooling is also handled by nqsserver, not sawserver. TOTAL_TIME_SEC is the time clocked once the request reaches nqsserver. Any wait for connections would happen before the request reached nqsserver. These would be reflected in the difference between START_TS and END_TS as mentioned above.

If you decide that server thread and dbgateway thread need to be adjusted, it is recommended that you get a site review from professional services.
Oracle recommends that unless you are experiencing a specific problem, that the settings be left at default.Problems involving sessions or thread ranges set too small would involve requests being queued. There are several settings that could come into play here including thread ranges and connections from connection pools.  Normally performance monitor counters would be set up to diagnose what setting is likely to be the bottleneck.As for the stack sizes, again you would need to be experiencing some sort of problem for Oracle to recommend a change.

Hope This Help

Sunil S Ranka

“Superior BI is the antidote to Business Failure”

Posted in OBIEE, sunil s ranka | Tagged: , , , , | 1 Comment »

 
%d bloggers like this: