Passing 300

It all began one summer

It seems so long ago (2,770 days or 396 weeks or 91 months or 7.6 years – but who’s counting?) that I first put pen to paper – Yes, I did.  Really.  I’ve now moved on to word processors for drafts and am thus so 21st century. – and started this blog.  And why the (re)counting?  Because this little corner of EPM inanity has hit 300 posts.  That’s an average of 39 posts of Stupid Programming Tricks, Compleat Idiot, Stupid Shared Services Tricks, Stupid Planning Tricks, and other sundry bits of EPM frivolity per year.  I pity you for reading this dreck.  Come to think of it, I pity myself for writing it at such a pace but on balance I think I feel worse for you.  

But it is a landmark of sorts and an opportunity to reflect on why this blog continues when so many contemporaneously launched blogs are moribund or nearly so.

So yes, 300 posts and yet some of you are still here.  Why?

Don’t know much about Essbase/PBCS/Planning/FDMEE/etc.

I seem to be forever chasing Oracle’s EPM seemingly ever-expanding products – how do I do X, how did someone else do Y (and how can I “borrow” their approach), why doesn’t that !@#$ing Z work?  Some of my fellow EPM practitioners seem to glide from tool to tool and solution to solution with nary a show of effort (Glenn, Celvin, TimG, TimT, Dino, and Pete I’m looking at each and every one of you.  With envy.).  I assure you that yr. most hmbl. & obt. svt never, ever, ever gets from A to B without a fair amount of pain.  Solving the problem is always fun, staring at it (best of course when in front of other people, the more senior the better) in complete incomprehension not so much.

So are you this?

Or this?

Everything I've Got Belongs To You

There are the greats in this industry – any industry really – and then there are the rest of us.  Is that so bad?  We’re not the smartest guys in the room but at least we get to be in the room.  Yes, I think I just insulted every one of you, Gentle Readers, but my point being that this blog’s primary purpose to help you and me get from A to B.  Maybe the fact that you read work-related blogs (obv. not just this one), read EPM books, follow EPM geeks on Twitter, and read and post on messageboards means that in fact you’re amongst the smart set.  Surely the smart ones use resources to solve their problems; surely the dumb ones don’t.  See?  I just rescued myself from having exactly zero readers.  Hopefully.

All kidding aside, this blog as it exists today would be pointless without you.  Thank you for putting up with what has been described as an idiosyncratic (read:  long winded with detours into obscurity) approach.  I hope you take the time to click on all of my laboriously-gathered links.  Goal one of this blog:  make you better EPM geeks.  Goal two of this blog:  make you all wish it was 1967 aka peak American popular culture as it’s a giant wasteland after that.  Let’s turn the clock back.  At least you’ll appreciate what your parents or grandparents (or in some cases great-grandparents) grooved to.

I’ve got your number

Google (Blogger and Google Analytics) is funny and by funny I mean inconsistent.

Here’s Blogger’s numbers:

Huzzah!  I’m closing in on a million page hits.

And then there’s Google Analytics:

Not-huzzah because it’s telling me that I’m closing in on half a million page views.  

It’s a riddle

A couple of interesting notes about the above:

1. People don’t read this blog around Christmas.  Not a huge surprise there.
2. My readership is going – slowly – down.  Why?

For the first, it’s nice to know that people have lives.

As for the decline (and it is real, alas) is I think based on two things:  number of posts per year (I hit my high in 2014 of 52 posts and readers vs. 40 the year after – less new content = less readers) and competition from other posts as well as Twitter and other social media.  I haven’t tried to count the number of EPM-related blogs extant today but it surely has to be about 50.  When I started it out the number was more like 10 although as noted most of those are dead, dead, dead.  YouTube, Facebook, and Twitter are yet more avenues for those who want to learn.

Or this blog sucks and is getting worse all the time.  You decide.

Why shouldn’t I

I like to think that actually the blog is getting better.  I’ve purposely hit on a combination of series posts such as the Compleat Idiot series on Planning in the cloud, Programming Stupid Tricks for unrelated Essbase, Planning, whatever-they-are tips and tricks, and community outreach posts such as live (sort of) blogging of Kscope, OpenWorld, and now meetups.  

You may have noticed that I’ve switched to a longer and more in depth approach in my Compleat Idiot cloud series.  There’s an awful lot to learn about Oracle’s cloud products.  Lots of innovation, yes, but also lots of work learning the tools and then keeping up with them.  I can’t think of how to do this except through this detailed way as so much innovation is coming out of the movement to the cloud.  Love the cloud or loathe it, money is being poured into the products in a way that simply hasn’t existed before.  That means the products change and expand constantly and that likely means the Compleat Idiot series won’t either.  That also means my life won’t get a lot better because some of these posts are over 50 pages when written in Word.  Ouch for both you in the reading and me in the writing.

While solutions to problems are what we’re all after, there is more to life and a career than code.  I’ve used this blog as a soapbox to encourage you in the strongest terms to get involved with our little community.  As an example, my involvement with ODTUG has utterly transformed my professional and personal life.  If it happened to me, it can happen to you.  Grasp the ring.  Reach.  Blow your horn.

Where I can, I’ve tried to also impart what little wisdom I’ve picked up in 20+ years of consulting in a 25+ year EPM so-called career.  Sometimes I shake my head at the folly of others when it comes to solutions (hubristically complex), code (ugly, hardcoded, slow, wrong – sometimes all four at once), and even social interactions (Is there anyone more awkward than a geek?  Thought not.) and then realize that I almost certainly did the same thing at one point or another.  Smart people learn from others’ mistakes.  Think of this as a plea to be smart and occasionally listen to me as I’ve made every mistake there is.  

The other bit of advice I’d give you is don’t be afraid to be a contrarian.  That of course doesn’t mean you’re always right, but reflect on why people say what they say.  Is a technical recommendation for the good of customer or is for the benefit of the speaker?  Is product X the solution that everyone follows because a vendor is pushing it or would some other simpler and cheaper approach work just as well?  

In a word:

 

Try to See It My Way

Have I fulfilled this blog’s mission?  Here’s what I wrote on 10 May 2009:

What about the “hacking” in the name of this blog? Hacking can mean all sorts of bad things and that’s what villains do. Good hackers are more interested in taking an ordinary tool (but so cool) and doing out of the ordinary things in a geek chic way.

To that end, I’m going to try to share with you some of the dumb things I’ve done and how you don’t have to do them, how to make Essbase do things it “can’t” do, and generally make Essbase dance.

Lastly and most importantly, I’ll also share code/techniques/approaches. I welcome your comments (constructive please, I have an average ego and it is bruised when pummeled) and most of all your suggestions for improvements. I’ve never written a piece of code that hasn’t been improved through examination by a fresh set of eyes and as a consultant if I can’t fix where I wrote it, I’ll make it better next time.

And, despite the title of this web site, I won’t limit the scope of my postings to Essbase. I’ll include anything else that touches Essbase, from Planning to Dodeca, to who knows what.

That, for good or ill, is pretty much what this blog is all about.  Through the passage of time I’ve forgotten about “geek chic” and shall henceforth casually drop it into conversation.

All kidding aside, I’ve tried very hard to live up to my vision of education and outreach and I think on balance I’ve managed to do it.

Watch what happens

So where does this blog go from here?  Will there be another 300?  Will I lose my ever-lovin’ mind and actually do this again?  Maybe.

So long as I’m involved in this little industry, I feel I have no choice but to keep learning.  Whether that’s through this blog, speaking at conferences, writing books, or in some other completely-monetarily-uncompensated form, I’ll keep on learning and sharing.  One day, hopefully not too (actually, yes, hopefully given what that entails) long from now, I’ll retire and this blog will come to an end.  I’m not dead yet and I’ve got a lot of livin’ to do so expect more of Cameron in one form or another.

Because of you

So yes, this blog exists because I use it as a mechanism to teach myself but making it public with a readership that rounds down to zero would be pointless.  Thank you for your support, your comments and corrections, and your continued readership.

Call me

Want to see a topic?  Have a question (hopefully) answered?  You can reach me care of this blog or via Twitter or via LinkedIn or reach out to me in person at meetups, Kscope, and Open World.

Be seeing you.

Stupid Programming Tricks No. 29 -- Dynamic Load Rule columns

Preface

This was originally going to be one ginormous post on Stupid Essbase Load Rule Tricks but once I got to page 12 and realized I was about halfway through I decided to split this subject into multiple posts.  At the end of it all I’ll put in links to bring everything into a single spot.

With the warning, off we go.

Hate is such a ugly emotion

It is moderately well know that Yr. Obt. Svt. hates Load Rules.  Why?  

I hate them because they have an interface that is little improved from the days of Essbase Application Manager.  If you want a feel for what that looked like, have a read through the DBAG which (un)surprsingingly hasn’t bothered changing the screen shots of what is after all fundamentally unchanged from 1993.

I hate them because they are a temptation to those without other data integration tools to manipulate data.  I really and truly have seen instances of over 240 Replace selections in a single rule.  How does one audit that?  Understand it?  Manage it?  Know that it’s actually performing per requirements?  The short answer is notgonnahappen.  Lest I be accused of casing stones at those less fortunate (eh, those of you who have had the (dis)pleasure of meeting me may think that’s a pretty low bar), that’s what the admim had at hand, that’s what he used.  But It Just Isn’t Right when a SQL INNER JOIN would have done the same thing and in a much better way.  But I digress.

And lastly, I hate them because I have comprehensively, completely, and totally shot myself in the foot, fired to slide lock, reloaded from my spare magazine, and repeated ad infinitum.  Seriously, it’s easy to do this even when one is being careful and, Best and Brightest, I’l bet you’ve done it more than once.  Ugh.

Another Load Rule rant over.  As always, it feels so good to vent my Load Rule spleen.

A plea for healing

And yet we need Load Rules.  The latest and greatest version of Essbase, Essbase Cloud Service (EssCS), that Tim German I presented at Kscope16 used – Wait for it! – Load Rules albeit in much improved fashion.  The need to get metadata and data into Essbase (and Planning) remains.  Until that golden day when all of we Essbase geeks are using EssCS or at least enjoying the functionality that it provides in on-premises Essbase we are well and truly stuck with them.  An even better alternative would be to  use the INSERT INTO…SELECT FROM data and metadata nirvana.  A man can hope.

And now a use case

And actually, they can be at least bearable if we only didn’t have to suffer through the interface and kludginess that is a Load Rule.  How can this be done?  Quite simply really – SQL is the answer.

My Very Favorite Essbase Message Board In The Whole Wide World (MVFEMBITWWW)

Over on Network54, there was a thread where the original poster (OP) wanted to dynamically change the number of columns he’d read from a table via a single Load Rule.  Huh?   And then there was the assertation that both overloading (or stacking) single data column as well as dynamic load rules aren’t possible.  Double huh?  That just isn’t so, or at least I don’t think that’s so.

It seems odd on its face but his requirement is to sometimes load history across multiple or a single year and/or sometimes load just the latest period.  It would certainly be possible to stick the year and the period in the column but this is a set of pretty big data at least in Essbase-land – potentially up to 3 BILLION rows.  Eeek.  

A far more efficient approach would be to stick the periods in the columns.  What is likely to be faster when loading three years of data in this format ~ 83 million rows (3,000,000,000 / 12 periods / 3 years) or 3,000,000,000 (the whole kit and caboodle)regardless of the number of columns?  Exactly.

Putting aside efficiency although I’m not clear on why one would want to do that, this approach would require multiple Load Rules – at least one for just a single period and one for multiple periods – and batch processes and then a selective execution of those Load Rules.  While this isn’t an impossible task, it’s certainly annoying.

Is there another way?  You betcha.

Stacking dimensions

The first step is figuring out how to put years into the column instead of the year.  Using Sample.Basic and (gasp) a text Load Rule and yes this is possible in SQL as you will see a bit later on. Let’s see how this might be done.

Typical and atypical

Typically each column in a load rule is a single dimension as shown below.  Easy peasey no big deasy.  When I build Load Rules and when I look at what others create this one-to-one relationship is standard.

That may be the default but it isn’t the only way to peel an egg.  You can stack or overload each column with multiple dimensions.  By that I mean a column that refers to measure Sales can also be directly tied to the scenario Actual while the next column can be Budget and Sales.

This can be built in a Load Rule by selecting two or more dimensions:

Note well the double quotes around the two member names.  These are important because they act as delimiters.  If not used, Essbase thinks that the string is a single member name of Actual,Sales.  What’s desired are two dimension assignments in one column.

When selected via double clicking on the member names the result is a data column definition looks like this:  "Actual","Sales"

Unpossible!

That’s all well and good but as you might imagine I prefer to have the data describe itself.  To do that I can create a data Load Rule that reads the first record as the header:

Without data, the Load Rule looks a bit empty:

By placing the header record as the first record, and using the above Data Source Property header row definition, the Load Rule magically has the stacked dimension definitions by column.  

NB – Essbase automatically trims double quotes so using an escape character is a requirement as shown below:

Maybe an easier way to view it is in Excel:

With that, I have a Load Rule that is Actual in columns 5 though 12 and Budget in columns 13 through 20.

For the record, escaping double quotes is performed using the backslash symbol as follows:

\"Actual\",\"Sales\"

Here’s what it looks like in a Load Rule column in EAS looking just as if you’d’ve selected it by hand.

Does it work?  Yep.

As always, the proof is in the delcious pudding.  I like South African Malva Pudding  but to each his own.

Dynamic SQL your way

It’s easier in a relational source.  Yes, that’s a drum I beat again and again simply because it’s true.

I can create a simple fact table with the stacked headers I need in the column names:

Note how SQL Server (one might think that I’d switch to Oracle sooner or later) sticks [ and ] around the special character double quotes.  There is no need for escaping those characters.  Why it is required in one place and not in another is the Sweet Mystery of Life.

A simple query returns the values:

Ta da:

And a load:

Good grief that was easy.  

Two cool things thus far

We now know:

1. Data columns can refer to more than one dimension on a column-by-column basis.
2. Either text files or relational sources can pass that multiple dimensionality either in a header record with escaped double quotes or via a column name in a fact table.
3. Never say never again when it comes to Essbase.

The next Stupid Trick in this series will show how to selectively turn columns on and off in a Load Rule.  

Be seeing you.