Why I hate (and love) Business Rules, part 2

The Two Minutes' Hate is over – the below is a paean of love to HBR.

Prequel to the cool stuff
So in part 1, I reviewed how to create a HBR, substitute in local variables to read a Planning form’s Run Time Prompts (RTP), and suggested that it might be worthwhile to read this post.

The code sample I used doesn’t do any fancy allocation or calculation. It just aggregates the database using form POV members.

You may be scratching your head, thinking, “Why on earth is he banging on and on about this? Just issue an AGG(“Entity”, “Segments”) statement and be done with it.”

Zoiks! If I wrote one line of code to consolidate a database I would be out of a blog post topic – This Is A Bad Thing.

Secondly, Why Would I Want To Do That? (Ex-Comshare employees/customers/partners know of whom I speak when I write that.)

Because thirdly, and most importantly to you, I can make that database aggregate much, much more quickly.

The Trouble with AGG
Other than sounding like a misquote from a sketch in “Monty Python and the Holy Grail”, what is wrong with an “AGG” when applied to the Entity and Segments dimensions?

Think about it in the context of the form as shown below:

If I change any of the data values on this form, I am only changing them for DVD Recorders in Pennsylvania. New Hampshire isn’t in the POV. Neither are eight-track cassettes.

But if I wrote:
AGG(“Entity”, “Segments”) ;
I would be also be aggregating a New England state and a totally obsolete electronic media, if there was any Gross Profit data for those two members. After all, AGG is kind of a blunt axe.

(For you Intelligent Calc lovers out there, yes, I could use the blocks’ clean or dirty status to not calculate upper level data that already exists, but good luck doing that in a real world Planning application. There will be clean/dirty blocks all over the place and it is very difficult to keep this approach from going pear shaped.)

Having disposed of the Intelligent Calc option, while an AGG statement will certainly come up with the right result, Why Would You Want To Do That when you only changed data at the PA and DVD Recorders intersection? Why aggregate data combinations for all of the other level zero Entity and Segment combinations that haven’t even changed?

(These questions do have a point, so bear with me.) You might answer, what else am I to do? How do I consolidate my dimensions if I don’t use AGG or CALC DIM?

Tricky, innit?
Here’s the trick – you don’t need to calculate the dimension, you only need to calculate the relevant hierarchies. That’s what the calc script above shows and what your HBR can do for your forms.

@IANCESTOR is your friend
@IANCESTOR is your BFF? That I couldn’t say. But it’s going to be your HBR pal from now on because, if you think like Essbase, you can make Essbase only calculate what you want. *This* is hacking Essbase.

Let’s review the way Essbase aggregates a database, per our dear friend, the Database Administrators Guide (DBAG). For the below section we’re going to ignore the Consol database to go along with the DBAG. Just substitute Entity for Product and Segments for Market; they are the first and second consolidating sparse dimensions.

How Essbase aggregates a database
Per the ever-scintillating DBAG, Block Storage Option (BSO) databases calculate dimensions in the following order (we are only going to concern ourselves with sparse, aggregating dimensions), “Sparse dimensions (in the order they display in the database outline)”. See http://download.oracle.com/docs/cd/E10530_01/doc/epm.931/html_esb_dbag/dcacaord.htm and the “Member Calculation Order” section if you need to cite chapter and verse.

Okay, we know the order that Essbase is going to calculate the dimensions. What about the order of the members within the dimensions? This is answered by the “Block Calculation Order” section :
“Essbase calculates blocks in the order in which the blocks are numbered. Essbase takes the first sparse dimension in a database outline as a starting point. It defines the sparse member combinations from this first dimension.”

Using My Very Favorite Essbase Database In The Whole Wide World (MVFEDITWWW), i.e., Sample.Basic, this means that:
“In the Sample Basic database, Product is the first sparse dimension…Product has 19 members…Therefore, the first 19 data blocks in the database are numbered according to the calculation order of members in the Product dimension.”

The DBAG goes on to say:
“The other sparse dimension is Market. The first 19 data blocks contain the first member to be calculated in the Market dimension, which is New York…The next member in the Market dimension is Massachusetts. Essbase creates the next 19 data blocks for sparse combinations of each Product member and Massachusetts.”

This is the important bit:
Essbase continues until blocks have been created for all combinations of sparse dimension members for which at least one data value exists.

Guess what, we have just reviewed how Essbase calculates sparse dimensions from the first sparse dimension to the last one, by dimension and within each dimension. In essence, block by block.

In plain English:
1) Product is aggregated for every level 0 Market member (where data exists in Market – Essbase is smart enough not to calculate combinations that don’t exist).
2) Then Market gets aggregated by every Product (that exists, there’s Essbase being smart again).

Okay, but so what?
If we jump back to the Consol database, you may recall that I claimed you don’t need to calculate all of Entity, do you? Nope. Just the member in the form POV and its ancestors.

You also don’t need to calculate all of Segments. No, sir. Just the Segment that is in the POV and its ancestors.

This is the trick/gimmick/optimization/clever bit/thing you maybe already knew long ago and are now totally disappointed by. Sorry if you’re in the last category – I swear the next post will be better.

How do you do this?
Simple. Let’s assume that the Entity is PA and the Segment is DVD Recorder. Remember, you only need to calculate the relevant branches of the hierarchy. The other level zero/upper level members haven’t changed, so there’s no profit in recalculating them.

Use @IANCESTORS in combination with FIX statements to make Essbase aggregate:
1) The PA ancestor tree for DVD Recorders.
2) The DVD Recorder ancestor tree for the PA ancestor tree.

The results of the above versus that AGG/CALC DIM is exactly the same, only the focused aggregation does it in less than a quarter of the time.

Correction

My good Hyperion buddy Joe Aultman pointed out an error, or at least a redundancy with the above code.

Basically, the @IANCESTORS within the FIX statement isn't needed as the members themselves don't need to be aggregated, just their ancestors.

The code as posted will result in the right value, but will be that fraction of a second slower as it is addressing two more blocks. We don't want that, right?

I don't know why I wrote it in this blog with the @IANCESTOR within the FIX as that isn't how I do it at my clients. Overthinking it, I guess.

I have inserted a snippet with the correct, @IANCESTOR-in-the-FIX-only approach below. This code also shows HBR local variables for the form's run time prompts.

Back to our regular programming

That’s it. Skeptical, are you? Can’t be that easy? The proof of the pudding is in the eating. Let’s trace the data by only changing Operating Revenue for January through March.
1) Here’s the form with the original data:

2) Let’s round the numbers up and send to Essbase:

3) Switching to Excel, the blue cells show the data that should be updated. This sheet is easy peasy, as it just shows aggregated dynamic Accounts.

4) And here it is where we expect to see it aggregated one level up, by Entity and Segment.

5) And now let’s look at the when the focused aggregation HBR is run.

Notice that it doesn’t matter if we do or don't aggregate MA, NY, DVD Player, Portable DVD, and DVD/VCR combo. Only PA and DVD Recorders changed, so only their parents need be aggregated. In other words, only aggregate parents whose children's values change, and leave the rest be.
6) Here it is rolled up by Entity.

7) And rolled up by Segment.

The Payoff
What does this mean from a performance perspective (faster=better)?

Looking at the Plansamp.log file, we can see that plain old AGG took almost 7 seconds:

Where the focused aggregation didn’t even make it to 2.5 seconds.

In the Real World

Just yesterday I benchmarked a real Planning HBR with the AGG versus focused aggregation approach. How about 180 seconds versus 24 seconds? Now we’re talking 1/8 of the time. This is powerful medicine.

YMMV; these are the results I got for my client’s Essbase database. Every database is different, so my performance improvement won't necessarily map to your application. Regardless, the above technique can make “big” Planning applications fast and give you the run on save performance your users demand.

See, I love Hyperion Business Rules.

See you next time.

The Fare Hike Is Unfair

Once again, politicians who pay lip-service to improving transportation are trying to put your money where their mouths are: Governor Rell is proposing a 10% fare increase for Metro-North and a 40% fare hike for bus riders.

Her arguments for raising fares are specious:

1) New York Raised Its Fares, So We Should Also: NY State is raising its fares to pay debt service on $12 billion in bonds raised to invest in subways and trains. It had threatened 23% fare hikes and draconian service cuts (even in Connecticut), so the compromise 10% fare hike (June 17th) seems like a bargain. The MTA dug itself into a financial hole and wants riders to dig it back out. And Connecticut should mirror such bad public policy?

2) The Special Transportation Fund Is Running Out Of Money: True, but this is because lawmakers stupidly lowered gasoline taxes a decade ago. Those fuel taxes help subsidize rail fares and I predicted then that their loss would lead to higher fares. The way to replenish the Fund is to raise gas taxes. For just a one cent per gallon tax increase the state would gain enough revenue to halt the planned bus fare hike.

Sorry, Governor. Your rationale for taxing commuters just doesn’t make sense. Consider the consequences of these proposed fare hikes:

1) Increased Road Traffic: Higher fares just encourage people to get back in their cars and drive on already congested highways. Isn’t this what we were trying to prevent?

2) Exploiting The Poor: The folks who take the bus don’t have cars. They have no other option than to travel by bus to school, to jobs and shopping. For them, a 40% fare increase means less money for food and medicine.

3) Discouraging Business: What employer will want to open a new business in a state where potential employees can’t afford to get to their jobs? A fare hike on trains and buses is anti-business and anti-growth.

4) Reduced Ridership / Even Higher Fares: Making the trains and buses more expensive will discourage ridership just as the new M8 cars start to arrive. Fixed operating costs won’t change, but reduced income from reduced ridership will just lead to calls for more fare hikes, a never-ending downward spiral.

5) Fare Increases Are Already Planned 2010 – 2016: Rail commuters already know they’ll be paying a 1.25% fare hike January 1st in 2010… and additional 1% fare hikes each New Years Day until 2016. This money is to help pay for the new M8 cars which are already behind schedule due to design problems and testing issues.

Hopefully, Governor Rell is just bluffing. Maybe she’s using the fare hike threat to jolt the legislature into action. But what politician would be so foolhardy as to support these fare increases, then look voters in the eye and ask for re-election?
While the downstate delegation may “get it” when it comes to supporting mass transit, the pols upstate clearly don’t have a clue.

Remember, it was just two years ago that Senate President Don Williams from “the quiet corner” of rural Connecticut was proposing free fares for senior citizens on all trains and buses. What a concept: a free ride on Metro-North for seniors while working stiffs pay $300 a month. Fortunately, that idea went nowhere.

So keep an eye on the legislature in the coming weeks. The process of creating a balanced budget won’t be easy or pretty to watch.

But if you want a say in stopping a hike on bus or train fares, contact your state lawmakers now. Only if bus and rail riders speak up can the Governor’s plan be defeated.