"Given the undeniable trend towards all-encompassing change in software development, the case can be made that general purpose software is doomed to always be unreliable and buggy."




"Is this some sort of collective insanity that has somehow woven its way into our society?"

23.7.08

 

Sharpening the Focus on ICIs

In order to show how mapping each item in a project to a ICI an make managing the project easier, it is necessary to define some type of structure that we can talk about. In this installment I will talk about the structure of an ICI and why it is so flexible.

As I have already pointed out, the first thing we know about an ICI is that it has a unique identifier. This can be in any form that you have a way of guaranteeing is unique. If you are a database-knowledgeable person, this would be equivalent to a unique key in a table. It must be entirely unique within a project. For the sake of simplicity, we will use an incrementing number for our examples, although in a real life situation I would opt for a Universally Unique Identifier (UUID) to guarantee uniqueness across projects in different databases. In addition to a unique identifier, what else do we need to manage an ICI? If you remember, we also talked about the need in IM to version everything. That implies that an ICI should have a version associated with it. It should. Each time the item that the ICI represents changes, the version should be incremented.

So far we have the following:

ID (unique identifier)
Version ID (incremented)

This is a good start, but that can't be all. I said that each ICI represents an item in the project. How do we know what that item is? Is it a .dll file? A configuration file? A SQL script? A user document? It seems that a good thing to know is what we are dealig with. Maybe I don't want to see all ICIs, only the ones that are user documents. Storing an indicator of what the underlying item is would be helpful in determining which ICIs I wanted to work with, so we should add a ItemType to our ICI.

So far, so good. We can uniquely identify this item, we can find out what version it is at, and we can get a general idea of what it is. But what if I want to actually get a list of the documents, not just a bunch of unique IDs? In this case, we would want to add a Source File value to it. But wait...what if I want to only see the user documents that are for the system administrator? Should I add an "intended audience" value also? That seems to make sense unless you consider that we could be talking about a .dll file instead of a document. Now "intended audience" makes no sense at all. We seem to be headed down a dead-end road.

If you take this discussion to its logical end, you will realize that everything that could possibly be tracked about ANY item in your project will need some place to put data. This littl ICI is quickly starting to turn into a massive data structure with potentially hundreds of data items. It is a database designer's nightmare! What to do?

If you think back to the last discussion, I talked a little about the powers of abstraction. By ignoring some things, or looking at them at a higher level, you can do some pretty amazing tricks that are not immediately obvious at the lower level. In some cases they are downright impossible. But if we reframe the problem, a solution will emerge soon enough. Lets pretend for a moment that we are not talking about a project, but a retail order. An order can have a single item, or several items. Come to think of it, an order can have an unlimited number of items. How it this normally handled? Skipping over the technical explanation, the answer is that at the order level, all items, no purchase what they are, are handled as "order items". Like ICIs, they can represent anything in inventory. With this approach, we don't have to worry about whether or not an individual item has a particular value because it is still an order item and can be treated like all others. While they may have some common pieces of information (stock number, quantity, price, etc.), the products that these pieces of information represent are really not very much alike.

Coming back to ICIs, we identified that there are several things that are common between all ICIs, but potentially many more that are not. The way to handle this seemingly impossible situation is to abstract this never-ending pile of possible data items down to a single thing: a Property.

Let's take a look at what that does for us. For our purposes, a property is defined as "a piece of information relevant to an ICI". It is obvious that we must be able to associate multiple properties to a single ICI, but once we come to that conclusion, the vexing problem that we were facing melts away. Now a .dll file can have certain properties (file name, size, install folder, etc.) while a web server can have other properties (Manufacturer, model, processor(s), RAM, etc.). By breaking apart properties from the ICIs that they are associated with, we have solved the problem of how to anticipate all possible pieces of information to be tracked. The solution is that new properties are created as they are needed. This solution, however, leaves a problem to be solved. On one side of the room we have ICIs and on the other side of the room we have a big pile of properties. How do we know which properties go with which items? This is a fairly easy problem to solve logically. With retail orders, each item has an associated "order number". The order number is unique per order (just as an ICI id is unique per ICI). When an item is added to an order, a line item record is created that contains both the order number and the product code for the item ordered as well as other pieces of information. Now we have a link that ties our item ordered to a particular order. To reconstruct the order, all we have to do is retrieve all line items that match the order number.

This is similar to what is done with an ICI. As properties are added for an ICI, they store not only the property data, but also the unique ICI ID.

A simple puzzle. You have a room full of kids and no information about who they are. You don't even know their names. You also have a list of parent names. How do you figure out who each child's parent is? The answer is pretty obvious; you ask the kids. The point is, the kids know who their parents are so you don't need any other method of figuring it out. in fact, you don't even need to know anything about the child, you just need to ask what their mom or dad's name is and find them on the list.

Similarly, in the ICI scheme we are discussing it is very easy to figure out which property goes with which ICI because the property "knows" who their parent is just as the order items "know" which order they belong to due to the order number "link" that each one contains. This is referred to as a "linked list" in programming and is a common method for linking a single item to multiple sub-items.

Armed with our new technique of defining properties as they are needed and linking them to an ICI, we can now put the problem aside of figuring out what an ICI looks like. It is sufficient to say that an ICI has a unique ID and one or more linked properties. Problem solved! We have come full circle and in the end, decided that the only thing we need in an ICI is a unique identifier value. Everything else can be linked to it as we need to add information. We will leave the actual "how" of linking properties to ICIs for a later discussion. Just trust me that it can be and is done every day in software that you probably use.

There seems to be a axiom emerging as we continue to discuss the methods we can use to create simplicity from complexity and order from chaos. You will see this theme emerging time and time again as we progress because it is a very simple and powerful technique. What it boils down to is "If you don't know the answer, try changing the question." Not something you are likely to get away with in other parts of life unless you are a politician, lawyer, or philosopher, but it works in Integrity Management quite well.

Now that we are mentally at peace with our ability to forge ahead without having to know and account for everything we may encounter along the way we can start looking at ways we can leverage this newfound power to make the job of creating software more reliable and easier.

[4]  Comments:

Blogger Emilian Bold said...

Could you write you blog in a more abstract way ? I think not.

Now, really, you are starting to say something (unlike the first posts which were more like teasers) but, please, try to make it shorter and denser. I don't want to grow a beard till you finish whatever you might have to say.

This is the second time I almost unsubscribed from your RSS. Next I'll just do it and maybe read the conclusions of your blog in some other form somewhere in the future (that is, if something will come out of it).

permalink24/7/08 03:17  
Blogger M@ said...

I am glad you have not unsubscribed, but I am not sure I understand your point. As far as I know, a blog is what I feel like writing about. Not all concepts can be effectively described in a paragraph or two, especially not concepts that are (as you have pointed out) abstract in nature. I do not plan out a schedule of what I am going to write and I feel that it is important to get the groundwork in place and build on it.

If this is too tedious for your taste, that is OK with me, but I am not going to change what I am doing to suit whoever comes along. It is obviously impossible to please everyone all the time.

I am writing about this because I feel that it is a useful concept that has been largely overlooked in the software development world and is one of the key factors in project failure. It may take a month or two years to get it all written...I don't know. If you can be content reading it as it is written, then great. If not, please do check back in a month or so and see how it is going.

permalink25/7/08 09:00  
Blogger Emilian Bold said...

First of all, thanks for posting the comment, I wasn't sure you will approve it.

Basically I get your point, you want to take it slow. But I'm not certain your slow style is due to being highly meticulous about it, or just building more and more hype while almost telling us what this largely overlooked thing is and how we can start actually doing something with this new discovery in building better software.

Maybe you are right, maybe it is complicated, maybe it's not one of those "learn this new thing in 10 days" but more like "learn it in 10 years". But then, perhaps a blog is not the best medium to say it. I'd suggest a book, then I can just skip forward and see what are you talking about and if it's of any use.

permalink25/7/08 09:08  
Blogger M@ said...

I do value people's input and comments and unless they are just downright nasty or off topic, I always publish them.

I also understand your point. You have no guarantee that I will actually get to a point, and my initial posts which you aptly described as "teasers" could reinforce that perception. However, I assure you that I am not just trying to build up traffic with teasers. You may notice that I have no advertisements or any other vested interested in "getting more traffic." I simply want to publish in an accessible medium what I have learned by banging my head on my monitor so many times.

Since you make good points and others may feel the same way, in my next post I will offer an overview of where this is heading. I hesitated to do this too early because without understanding the structure of it, many people will simply decide that it is another high-minded pointless rehash of the same old ideas. I don't believe that it is, and didn't want to sound like I was making claims that couldn't be true. But I think you are right. The time has come to pull the cover off this thing and let you see the whole thing. But be warned...after that I am going back down into the details.

permalink25/7/08 09:38  

Post a Comment

<< Home