Digging in to Project Integrity Management
What is "Project Integrity Management"? How is it different from Configuration Management? How does it make it any easier to prevent the dreaded project meltdown? I'm glad you asked.
As mentioned in a previous article, Configuration Management attempts to control elements of a project, specifically controlling and managing the constant change of information associated with the project. The amount of information to be tracked can be overwhelming, and the task soon becomes too complex to be performed manually. This leads most projects to seek a "toolset" to manage their configuration for them. While many commercial and some open source toolsets exist today, they all seem to be solving different problems. This is most likely due to the variations in different people's understanding of what Configuration Management consists of.
What many projects call CM is actually not configuration management at all, but simply source code control. For this task, there are a number of software packages available that provide varying degrees of functionality and frustration. But source code control is not configuration management. I would even go so far as to say that it is not even part of the configuration management process, but I will save that argument for a later time when I feel like starting a flame war.
At this point you may be thinking "this guy is either crazy or high"! I assure you that I am neither. Read on.
In the interest of avoiding splitting hairs on what Configuration Management is or is not, I have opted to create a new term altogether that will represent the concepts that I feel are the ones that need to be defined in order to get a handle on the mess that many of us know as the "development project". This term is Project Integrity Management and although it is similar in some ways to Configuration Management, the two are not interchangeable, at least not with the definitions I have seen of CM. So from here on out we will be speaking of IM instead of CM unless I am referring specifically to a Configuration Management concept.
First of all, IM is not source code control. This has been done many times over and, although I don't think that the approaches that are currently being used are great, I have nothing to add to that debate. It isn't that I don't think source control is important, I just think that the process is well understood and we should focus on some of the more vexing problems that have not been so thoroughly studied and developed for.
At the highest level, IM consists of the two following areas:
Versioning
Environment Management
These two concepts represent everything needed to completely track and control a project's variables. As you may have guessed, there are many lower levels to these concepts which we will explore, but first I would like to describe what is meant by these two terms.
Versioning is the most important concept in IM. Without versioning, the rest is impossible. In IM, everything is versioned. If it is not versionable, we are not interested in it because the definition of "versionable" is "possible to change". If something is impossible to change throughout the life of a project, then why would we care about it? It is in an eternally known state and tracking it would be silly. What would we track? The catch in this definition is the very important word "possible" is a pretty wide-open one. For example, you may consider that the operating system that is being developed against is not going to change during the project, but you would likely be wrong. Service packs change operating systems all of the time. Servers can crash and be replaced with different machines. Documents can be edited, corrected, and revised. The upshot of this is that pretty much everything associated with a development project can change. Nothing is "set in stone". What that means in IM is that everything must be versioned. Yes, that includes operating systems, servers, and documents. If it can change and it is part of a project (i.e. part of a deliverable), it must be versioned. We will explore the finer points of versioning soon, but for now this short discussion will serve our purposes.
The other high level concept in IM is Environment Management. This may or may not be a new term to you; I didn't make this one up! But I am going to heavily redefine it to suite IM's purposes. What I am referring when I mention Environment Management is the control of all versioned items in each project environment. Examples of environments in this context are "development", "test", "training", and "production". However, as you will find out, this is not to say that there is only one development or test environment. In most projects there are several. The IM approach to Environment Management is one of "Just In Time" environment creation and destruction. Environments are like Krispy Kreme doughnuts. As soon as they are created, they start going stale. I will also provide a detailed approach to implementing this JIT environment control plan.
Those are the high level concepts. On their own they are not very useful though. In fact, at this point, they may not even be comprehendible to any meaningful degree. But if you could see the picture in my head (which I hope to bring you around to doing), you would see that not only is it comprehendible, but very elegant and flexible. Now that we have gotten the mile-high view, I want to jump to the other end of the spectrum and give you a peek into how this can be pulled off. But first, a quick trip to the store.
Some of you older folks may remember a time when grocery stores put price stickers on everything. Each can of green beans or salad dressing had a white or orange sticker on it with a price. No barcode, just a simple number like "$1.29". When you checked out, there was a cashier who could (brace yourself) actually type numbers into a cash register. In fact, she could typically type them very fast with one hand and move the groceries to the bagging area with the other.
One day I noticed that some things that I bought at the grocery store had a new "built in" tag with a lot of lines on it. It seemed completely useless to me because the only thing on it besides the row of lines was a seemingly random series of numbers. I eventually learned that these were called "UPC" or Universal Product Code tags. Since the tags were implemented long before most grocery stores received and installed the cash registers that could read these tags, they seemed rather pointless to most people. But soon all of that changed. Because each product carried a unique code, and that code was the same for that product whether it was on a store shelf in Akron Ohio or Pascagoula, Mississippi, products could be paid for, inventoried, and ordered all using the UPC number. In fact, those processes eventually became completely automated. Modern checkouts at large chain stores such as Wal-Mart or Target read the UPC code, look up the price, subtract a single item of the matching type from inventory, and if the supply is getting low, automatically order new stock. There are currently some experimental stores that constantly monitor supply and demand and adjust the price of each item in the store based on these real-time statistics!
It is difficult to overstate the far reaching impact the simple UPC has had on the retail industry. It has enabled stores with millions of items in stock to instantly know how many of each item is on the shelf, how many are on order, and how many were sold in the past hour.
That is all very interesting, but what has it go to do with what we are talking about? I am about to tell you!
Before the days of UPC, the task of pricing, inventorying, and ordering stock was very time consuming. Since the cashier only had the price that was paid for an item to enter (not what the item was), there was no way to know how much the inventory of a particular item had been depleted other than manually counting items or, more commonly, "eyeball ordering". A department manager ordered new stock when the stock on the shelf looked low. In many ways, the problems they faced had a lot in common with the problems we now face on software projects. We have a huge number of individual pieces which must be assembled in just the right way to work. To make things worse, they are always changing. It would be as if there were gremlins in the store that constantly rearranged the shelves when nobody was looking and mixed different items up. Actually, I think those really did exist. They were called kids. I know because I mixed up a few shelves myself. With this overwhelming amount of information to track and with many, many items changing all the time, something is needed to get the kind of control on project pieces in a similar way UPC codes allowed stores to get control of inventory. We need a UPC for our project pieces.
Enter the Integrity Control Item. This is a concept that treats anything that is versioned (i.e. everything we care about) as a unique item. Similar to the way a can of Green Giant whole kernel corn has a code that is unique throughout the food supply system, our ICI is unique throughout our project. Actually, properly implemented it is globally unique even between projects, but more on that later. For now it is sufficient to say that a ICI is unique in a given project. In our examples, I will just use a numeric value like "324398675" to represent an ICI.
So what does that do for us? Great. Now instead of worrying about the Oracle Client drivers, I can worry about keeping up ICI #324398675 and its thirty eight thousand brothers. Yea, buddy...that sure made my life easier! But wait...there's more!
If the only thing we got out of an ICI was a hard to remember number it would be no better than a cell phone, but that is not the case. When a grocery store orders something with a UPC code, when the order comes in, it has additional information with the UPC code. For example, it may have the manufacturer, the product name, the quantity, and the country of origin. You don't need to know the country of origin to order the corn, you just need to know the UPC code, but if you need to know what that UPC represents, a large amount of information about it is available in the inventory control system. Similarly, if we want to automatically check that an ICI is present in a particular release, we don't need to know what it is or what its characteristics are ahead of time, those can be looked up when they are needed or not looked up when they are not. There are many, many other advantages to tracking everything in your project with an ICI which will become more obvious as we dig deeper into this process, but for now I will let you ponder these ideas and try to shoot holes in them.
Next time I will discuss how versioning works with an ICI. I hope you come back to read about it.
-M@
As mentioned in a previous article, Configuration Management attempts to control elements of a project, specifically controlling and managing the constant change of information associated with the project. The amount of information to be tracked can be overwhelming, and the task soon becomes too complex to be performed manually. This leads most projects to seek a "toolset" to manage their configuration for them. While many commercial and some open source toolsets exist today, they all seem to be solving different problems. This is most likely due to the variations in different people's understanding of what Configuration Management consists of.
What many projects call CM is actually not configuration management at all, but simply source code control. For this task, there are a number of software packages available that provide varying degrees of functionality and frustration. But source code control is not configuration management. I would even go so far as to say that it is not even part of the configuration management process, but I will save that argument for a later time when I feel like starting a flame war.
At this point you may be thinking "this guy is either crazy or high"! I assure you that I am neither. Read on.
In the interest of avoiding splitting hairs on what Configuration Management is or is not, I have opted to create a new term altogether that will represent the concepts that I feel are the ones that need to be defined in order to get a handle on the mess that many of us know as the "development project". This term is Project Integrity Management and although it is similar in some ways to Configuration Management, the two are not interchangeable, at least not with the definitions I have seen of CM. So from here on out we will be speaking of IM instead of CM unless I am referring specifically to a Configuration Management concept.
First of all, IM is not source code control. This has been done many times over and, although I don't think that the approaches that are currently being used are great, I have nothing to add to that debate. It isn't that I don't think source control is important, I just think that the process is well understood and we should focus on some of the more vexing problems that have not been so thoroughly studied and developed for.
At the highest level, IM consists of the two following areas:
Versioning
Environment Management
These two concepts represent everything needed to completely track and control a project's variables. As you may have guessed, there are many lower levels to these concepts which we will explore, but first I would like to describe what is meant by these two terms.
Versioning is the most important concept in IM. Without versioning, the rest is impossible. In IM, everything is versioned. If it is not versionable, we are not interested in it because the definition of "versionable" is "possible to change". If something is impossible to change throughout the life of a project, then why would we care about it? It is in an eternally known state and tracking it would be silly. What would we track? The catch in this definition is the very important word "possible" is a pretty wide-open one. For example, you may consider that the operating system that is being developed against is not going to change during the project, but you would likely be wrong. Service packs change operating systems all of the time. Servers can crash and be replaced with different machines. Documents can be edited, corrected, and revised. The upshot of this is that pretty much everything associated with a development project can change. Nothing is "set in stone". What that means in IM is that everything must be versioned. Yes, that includes operating systems, servers, and documents. If it can change and it is part of a project (i.e. part of a deliverable), it must be versioned. We will explore the finer points of versioning soon, but for now this short discussion will serve our purposes.
The other high level concept in IM is Environment Management. This may or may not be a new term to you; I didn't make this one up! But I am going to heavily redefine it to suite IM's purposes. What I am referring when I mention Environment Management is the control of all versioned items in each project environment. Examples of environments in this context are "development", "test", "training", and "production". However, as you will find out, this is not to say that there is only one development or test environment. In most projects there are several. The IM approach to Environment Management is one of "Just In Time" environment creation and destruction. Environments are like Krispy Kreme doughnuts. As soon as they are created, they start going stale. I will also provide a detailed approach to implementing this JIT environment control plan.
Those are the high level concepts. On their own they are not very useful though. In fact, at this point, they may not even be comprehendible to any meaningful degree. But if you could see the picture in my head (which I hope to bring you around to doing), you would see that not only is it comprehendible, but very elegant and flexible. Now that we have gotten the mile-high view, I want to jump to the other end of the spectrum and give you a peek into how this can be pulled off. But first, a quick trip to the store.
Some of you older folks may remember a time when grocery stores put price stickers on everything. Each can of green beans or salad dressing had a white or orange sticker on it with a price. No barcode, just a simple number like "$1.29". When you checked out, there was a cashier who could (brace yourself) actually type numbers into a cash register. In fact, she could typically type them very fast with one hand and move the groceries to the bagging area with the other.
One day I noticed that some things that I bought at the grocery store had a new "built in" tag with a lot of lines on it. It seemed completely useless to me because the only thing on it besides the row of lines was a seemingly random series of numbers. I eventually learned that these were called "UPC" or Universal Product Code tags. Since the tags were implemented long before most grocery stores received and installed the cash registers that could read these tags, they seemed rather pointless to most people. But soon all of that changed. Because each product carried a unique code, and that code was the same for that product whether it was on a store shelf in Akron Ohio or Pascagoula, Mississippi, products could be paid for, inventoried, and ordered all using the UPC number. In fact, those processes eventually became completely automated. Modern checkouts at large chain stores such as Wal-Mart or Target read the UPC code, look up the price, subtract a single item of the matching type from inventory, and if the supply is getting low, automatically order new stock. There are currently some experimental stores that constantly monitor supply and demand and adjust the price of each item in the store based on these real-time statistics!
It is difficult to overstate the far reaching impact the simple UPC has had on the retail industry. It has enabled stores with millions of items in stock to instantly know how many of each item is on the shelf, how many are on order, and how many were sold in the past hour.
That is all very interesting, but what has it go to do with what we are talking about? I am about to tell you!
Before the days of UPC, the task of pricing, inventorying, and ordering stock was very time consuming. Since the cashier only had the price that was paid for an item to enter (not what the item was), there was no way to know how much the inventory of a particular item had been depleted other than manually counting items or, more commonly, "eyeball ordering". A department manager ordered new stock when the stock on the shelf looked low. In many ways, the problems they faced had a lot in common with the problems we now face on software projects. We have a huge number of individual pieces which must be assembled in just the right way to work. To make things worse, they are always changing. It would be as if there were gremlins in the store that constantly rearranged the shelves when nobody was looking and mixed different items up. Actually, I think those really did exist. They were called kids. I know because I mixed up a few shelves myself. With this overwhelming amount of information to track and with many, many items changing all the time, something is needed to get the kind of control on project pieces in a similar way UPC codes allowed stores to get control of inventory. We need a UPC for our project pieces.
Enter the Integrity Control Item. This is a concept that treats anything that is versioned (i.e. everything we care about) as a unique item. Similar to the way a can of Green Giant whole kernel corn has a code that is unique throughout the food supply system, our ICI is unique throughout our project. Actually, properly implemented it is globally unique even between projects, but more on that later. For now it is sufficient to say that a ICI is unique in a given project. In our examples, I will just use a numeric value like "324398675" to represent an ICI.
So what does that do for us? Great. Now instead of worrying about the Oracle Client drivers, I can worry about keeping up ICI #324398675 and its thirty eight thousand brothers. Yea, buddy...that sure made my life easier! But wait...there's more!
If the only thing we got out of an ICI was a hard to remember number it would be no better than a cell phone, but that is not the case. When a grocery store orders something with a UPC code, when the order comes in, it has additional information with the UPC code. For example, it may have the manufacturer, the product name, the quantity, and the country of origin. You don't need to know the country of origin to order the corn, you just need to know the UPC code, but if you need to know what that UPC represents, a large amount of information about it is available in the inventory control system. Similarly, if we want to automatically check that an ICI is present in a particular release, we don't need to know what it is or what its characteristics are ahead of time, those can be looked up when they are needed or not looked up when they are not. There are many, many other advantages to tracking everything in your project with an ICI which will become more obvious as we dig deeper into this process, but for now I will let you ponder these ideas and try to shoot holes in them.
Next time I will discuss how versioning works with an ICI. I hope you come back to read about it.
-M@
[0] Comments:
Post a Comment
<< Home