Spreadsheet Design

The MIE Podcasts – Charles Williams

Charles Williams is probably best known for his work in Excel calculation speed and VBA performance, not to mention the worlds most famous Excel Addin Name Manager. In truth, he’s a expert in many areas of Excel and on top of all this he’s a great bloke. In this podcast we got down and dirty with Excel performance, uncovering some interesting insights along the way. It was a true pleasure talking with Charles.

Charles the First - King of Speed!

Charles in the founder of Decisions Models, a consulting firm specialising in Excel development, and the farther of the FastExcel addin, we talked about this addin and how it works and also his addin loader tool. This is one for the archives!

Thanks to Charles for making the time and sharing his knowledge with us all.

A rant on Excel and Bad Ideas.

The great thing about the internet is that anyone can share their thoughts, the problem with the internet is that anyone can share there thoughts. A classic dichotomy if you will.

We see this all the time, people talking utter, utter rubbish about things they don’t understand. As “organisations” have started to blog more and more this issue has increased. The way I see it is like this… If you run “Mega Corps” would you want me bloging for you? I probably wouldn’t. You see I’m not a talented writer, my spelling and grammar are poor and I’m not adept at making my point – as I will now prove. So “Mega Crops” ask the people who can do these things well to blog for them. What actually makes for a good bloger is someone who can think well! See, a bad idea expressed well is still a bad idea. What makes a great bloger is someone who can think and write well – and these types are rare. But still, what would you rather have, an ill-conceived observation expressed elegantly or a quality observation just expressed?

Which leads me to these two blog post, the first one suggests that people who use Excel in the supply chain are doomed, and the second one adds more fuel to that fire.

Beware Supply Chain Excel Users—YOU are DOOMED!!!!

Excel doesn’t excel in all cases…

Now I need to be careful here, I don’t what to offend anyone.

Equally I’d like to address the lack of understanding and insight, or at least offer a counter view. I have some grounds to make these observations, being somewhat familiar with Excel and having worked in supply chain for the last decade or so.

Basically both articles point out the limitations of Excel, principally by comparing the differences between Excel and a Generic ERP system. For example:

“Excel has features that can calculate safety stock by using prebuilt basic formulas such as moving average, standard deviations forecasts etc. Keep in mind that by doing this, an organization will not have the ability to see the entire supply and demand relationship. On the other hand, when the same safety stock is calculated within an application, it will provide visibility to outstanding supplies that need to be replenished; orders can be planned for production; and it can calculate how quickly organizations can turn a forecast into a deliverable product.”

Khudsiya Quadri, Technology Evaluation Centers

And

“I completely agree with the author that there is a big risk to SCM Professionals who rely too heavily on Excel.  There are all the reasons listed in the article such as  lack of collaboration, visibility, control and no ability to perform “what-if” scenarios.  I would like to add some additional thoughts to this discussion.”

Monique Rupert,  21st Century Supply Chains

So, Excel has limitations creating what if scenarios, and pre built basic functions!! Clearly not Excel experts! Another common theme is that Excel is not very good for collaboration, information sharing and data security…

It is almost impossible to control the integrity of spreadsheet data and access to the spreadsheet.  With multiple people accessing the spreadsheet and no security, how can anyone have any confidence in the data?

Monique Rupert,  21st Century Supply Chains

Well true. Sharepoint (etc.) might help here, but that’s another issue. So basically don’t use Excel as a database. Fair enough but…

Comparing Excel to an ERP system is like comparing a Motor Bike to a Train Network. It’s just stupid. Telling people not to use Excel because it does not have the same capabilities as an ERP system is likewise really bad advice. Excel is different to an ERP system, we could easily re frame the argument the other way around and draw the conclusion that ERP system are the work of the Devil!

In fact, lets do just that.

Beware Supply Chain ERP Users—YOU are DOOMED!!!!

ERP systems are slow expensive complex beasts, with poorly documented calculation methods, inflexible font ends, and limited reporting capabilities. I recently talked to a number of supply chain professional and was shocked by how many of them are using their ERP systems in blind faith that the system are optimised for their needs. In the fast passed global supply chains of today, how can these default settings and calculation models possibly be right for your business?

Consider yourself doomed if you ever find yourself using an ERP system for any of these:

Reporting and Analysis: ERP system can out put reports in a number of formats, but typically they cant build well constructed dashboards, which are tailored to your companies specific needs, and/or ones that can be quickly adapted and changed over time as the needs of the business change. And forget it if you want to do some sort of analysis that the System Architect didn’t think you’d need to do in the 2 month he was specifying the system for your company 3 years ago. (Not that that would ever happen of course…).

Changing the model: ERP systems provide various forecasting tools, is getting the best results from them a skill or a science? Do you even know what equations are being used, it’s unlikely because this is the IP of the vendor, and what about data sources, is that data from the Spanish plant accurate, if it’s not can you do anything about it? Not all that flexible are we!

Your team just grew: Better get your wallet out…

You want someone form out side the organising to use the system: No.

You have a new data source that you’d like to add to the model: Humm, can you see where we’re going…

And so on…

In conclusion neither ERP systems or Excel can doom your supply chain and comparing one against the other as a way to high light a weakness is a logical flaw. Neither ERP systems or Excel are inherently good or bad, badly designed and/or used spreadsheets are a business risk, just like a poor quality ERP system or ones that are used poorly are a risk. Each “system” has strengths and areas where their use makes sense, and this are extremely well document. Likewise the “miss use” of Excel and the motivations for this are also well document, and note – not all of them are unreasonable.

I take exception to the two blog posts because as well as misrepresent the capabilities of Excel, they also give poor or misleading reasons as to why it’s use is dangerous, also they both take one or two valid, but already extremely well understood points, and extrapolate them to reach irrational end points.

So were doe this leave us?

Well, here are some of my thoughts on what you should do to help you avoid risks if your using spread sheets (for anything really)

  • Design you spread sheets well from the ground up (here, and here for help)
  • Understand connected or discounted data sources
  • Learn how to connect to enterprise data bases.
  • Learn about versions, and version control and try to apply it
  • Think about risk, and except it explicitly if you have to

And there ended the rant ;-).

Spreadsheet functional programming

In his recent paper Spreadsheet functional programming, Dave Wakeling discussed the idea of using a functional programming language from within a spread sheet.

It’s an interesting approach, because the implementation effetely allows you to write code (in this case Haskell) in cell comments, in such a way that the results are computed on the fly – i.e. not complied.

As you might expect, there are a number of issues with the implementation from a practical stand point, but that’s not what I’m interested in here – I want to talk about the concept of functional programming in relation to spread sheets, which this paper also focuses on.

It’s my view that a spreadsheet is a functional programming language. This means that I have an issue with the goal of the paper, which is:

“Our hope is that by doing so, we might get spreadsheet programmers to give functional programming a try”

Nothing wrong with that, but my views that spread sheet users already are functional programmers!!! So why them might we want to use Haskell with Excel? Towards the end of the paper Wakeling suggest some possible strengths to this approach.

1. It uses an ordinary spreadsheet. The functionality of, experience with, and Support for a familiar product all carry over.

2.  It uses an ordinary functional language. The benefits of such languages also carry over. (Backus, 1978; Turner, 1982; Hughes, 1989).

3. It can accommodate any pace of change. The old way of doing things can coexist with the new one.

Unfortunately I don’t agree that these are real benefits. Oh dear! – Why?

1. This is a case for anything that  integrates with Excel; the benefit is independent of what it is that’s being integrated.  This is the benefit of using a spread sheet and a programming language over using a programming language on it own.  It’s tenuous to claim it as a strength.

2. The benefits are true of Functional v’s Imperative programming languages (lasy, high order functions). Since Excel functions operate in a very similar way to functional programming these benefits don’t really exists between Excel worksheet functions and a functional programming language.

3. This statement is true of any situation where the 2 technologies can co-exist you could say the same about using C, VSTO etc, etc, again it’s really tenuous to claim it as a strength of any like for like approach.

So if these strengths are not in fact all that real, then why would we want to use a functional programming langue with spreadsheets? Perhaps a clue is given in this passage:

“On the positive side, many everyday programming errors are detected that would otherwise lead to incorrect results, including dangling cell references, undefined functions, and functions with the wrong type of arguments.”

I can see how using a more “strongly typed” language could possibly help reduced type errors, not sure how undefined functions would be helped or how “dangling cell references” (in fact, I don’t know what this means at all) would be effected.  Generally further discussion and justification of the benefits of using a functional language with a spreadsheet are needed to build a case for doing so.

The paper did not convince me that using a functional language has any really benefits to using worksheet functions or any other programming language. Having said that, the paper does show that its possible to link technologies like this and it’s another interesting approach to programming spreadsheets. The prospect of F# and XLDNA and F# in VSTA/O are also interesting things to start thinking about!  The application of functional programming languages within spreadsheets is interesting  and Wakeling should be congratulated for opening up the debate for future discussion– as hopefully this post will do too!

Unit testing Excel VBA – xlUnit demo

Back in February I did a podcast with Mike Woodhouse, based around testing VBA code. Mike has written a unit testing frame work for VBA code, called xlUnit. Over the last few weeks I’ve been using xlUnit to help write a generic validation class for VBA, so I thought I would put together a little video of how the addin can be used to write tests for your code.

Its in 2 parts because you can only upload videos that are 10 minutes long to you tube.  The first part is the basics and in the second part I show some examples from my project.

You can find out more about xlUnit at Mikes blog, grumpy old programer:

The MIE Podcasts – Chandoo

Purna Duggirala. Purna Duggirala. Purna Duggirala. You might better know him as Chandoo, but his name is Purna Duggirala. Chandoo is a very friendly fellow, and we had a great chat about charts and dashboards, how to design them what approaches to take etc etc… So if that sounds like your sort of thing, have a listen.

The PHD company christmas card circa 2009......

Purna runs a great Excel site Pointy Haired Dilbert, it really is super, but I reckon that anyone who reads this will know that much already. He has an MBA from IIM Indore and Computer Science degree from Andhra University, and he’s an Excel MVP, of course!  only the best for the readers (and listeners) of Methods In Excel.

One thing I didn’t know about Purna is his ability to multi task, the bat eared amongst you will be able to detect Chandoo simultaneously talking and typing away… no doubt writing some more excellent blog posts for PHD!