In order to describe some basic software engineering concepts I will provide with a set of scenarios and recommendations which if followed should make most (never say all) software projects a success.  So why most software products and projects are a failure?  Read along and you should find the answers for yourself.

Software development is both and science and an art.  As a matter of fact, most (as was previously stated, never say all) engineering disciplines are both a science and an art.  For example, in civil engineering, an architect might design a structure (i.e., building, bridge, etc) which should be strong enough to support the stress and fatigue of it’s intended use and last for the expected / specified lifetime.  Structures are designed and built differently based on location.  A four-lane bridge for general traffic (automobiles, trucks) is designed and built differently when the intended location is Southern California than when its location is somewhere in Maine.  One of the sites the bridge is prone to experience rather warm temperatures, lots of traffic, earthquakes, strong winds, etc, etc, etc.  The counterpart in Maine might expect lower temperatures and perhaps salt corrosion if the structure is near or on the water.  Both bridges will share some basic requirements but they also have unique ones.  Being a layman in civil engineering, I can understand how requirements for structures will differ.  That does not make me a civil engineer nor gives me the ability to design one.

Algorithms are at the core of any software development project.  You use your favorite web browser and look up the definition.  Most people would just type the word “algorithm” and follow some of the links returned by the search engine of their choice.  I tried Google and at the top of the search I found a link to Wikipedia and a dictionary.  A dictionary tends to provide a basic definition that is good enough to understand the term (e.g., a set of rules for solving a problem in a finite number of steps, as for finding the greatest common divisor).  If you look at Wikipedia you may find something like “An algorithm is an effective procedure for producing a well-defined function”.  You need to realize that Wikipedia contents are generated by contributions of many people who in most cases do not have the experience or background or have some agenda to contribute.  Personally I seldom look at the contents of Wikipedia.  If I do, I check references and then narrow my search to reliable sources (typically papers and books from accredited universities).  I have taken several courses on computer algorithms and have also read several books on the subject.  It just happens that this quarter I have registered to attend a computer science course on advanced algorithms at the University of Minnesota.  The book for the course is “Algorithms” third edition by Thomas Cormen, Charles Leiserson, Ronald Rivest and Clifford Stein.  I just happen to have contacted Dr. Rivest years ago when I developed the first content addressable storage regarding possible drawbacks in the use of MD5 digests to be used as names / handles for files.  Dr. Rivest is an expert in software encryption and hashes.  The book defines an algorithm as follows:  “Informally, an algorithm is any well-defined computational procedure that takes some value, or set of values, as input and produces some value, or set of values, as output”.  Hopefully you are able to see the nuances in the definition and be motivated to delve further into the subject matter with the invitation by the initial definition presented in the book.  Please note that by reading a good textbook you will not become an expert in the subject.  You also need the experience and hopefully you will be able to come up with better than average elegant approaches (algorithms) to solve software issues that you may be confronted with in a project.

MapReduce is allegedly the mechanism / technology / algorithm used by Google to manage huge amounts of data.  Allegedly Google has developed a proprietary mechanism (Google’s patent #7,650,331) to manage large sets of data using a distributed architecture.  Google does not make use of a SQL database (i.e., SQL Server, MySQL, etc).  Apparently Google uses no-sql.  By reading books that describe to some extent the technology, MapReduce is nothing else that the implementation of the divide-and conquer paradigm that has been used for decades in multiple algorithms such as merge sort.  Being able to file a patent does not require an elegant or unique algorithm.  As the Google patent demonstrates, one is able to patent an old idea just applied to a different scenario (in this case a distributed system).  Given that America is becoming a very litigious society, it is the best interest of companies to be able to fill their shelves with patents.  The ability to do so tends to be a function of the quality of the methods / processes developed by software engineering teams.

Computer Science (CSci) as its name states is a science backed up by mathematics.  CSci is also considered art.  Once again, going back to a dictionary, art is defined as:  Works produced by such skill and imagination.  When viewed under this definition, CSci requires both education and experience.  We all have reached some level of formal education.  When studying any science or mathematics, we first read the textbook and attend lectures on a subject.  In order to solidify our knowledge we perform some exercises and tend to solve some problems.  Sooner or later in our professional careers (assuming you exercise in some technical, scientific or mathematical field) we have encountered problems that due to our education and experience have been able to recall one or more techniques that when applied to the situation at hand, tend to solve the issue.  Based on education and experience some solutions are more efficient (consume less resources) and elegant (simple to describe, implement and maintain) than others.  I have made on multiple occasions reference to the popular idiom  “There is more than one way to skin a cat”.  As I always say, “yes there is, but of the entire possible set, there is one that is worst and one and only one that is best”.  The ability to most of the times produce solutions that lean towards the best side is the result of education, experience and ability which can easily be described as art.

 “The Art of Computer Programming” is what one may say another example that computer programming is not only a science backed up by mathematics, but also an art in itself.  Donald Knuth has written (and as far as I know he is still doing so) a collection of books that deal with the description and analysis of may computer-based algorithms.  Professor Knuth started work on the book set in the 1960’s.  As I recall in the 1980’s most respected software developers and computer scientists frequently referred to the works of professor Knuth searching for algorithms that could be used when confronted with a new design.  I have owned a couple copies of “The Art of Computer Programming”.  The main reason for this is that the set has been growing through the decades.  I currently have this set of book at home on a desk I use to study, read and think.  Perhaps this is an indication that I am not a young pup.

Most of us at several points in time (some more than others) get ill and have the need to seek medical attention. Depending on the severity of the ailment, we make an appointment over the phone with a clerk.  The day of the appointment we show up and a receptionist collects information and typically the co-payment for the visit.  After a few minutes a nurse leads us to a consulting room making a quick stop by the scale.  After taking our temperature and blood pressure we are asked a few questions as to why we are interested in seeing a general / family medicine practitioner.  Finally we see a physician.  If the ailment is obvious the physician prescribes some medicine and out we go to the pharmacy.  In most cases after a few days we are feeling well.  If the issues affecting us are not than obvious, we get sent to the lab for a battery of tests.  Typically some tests are of chemical origin (sugar level, white cells count, etc) while others may be imaging (i.e., x-ray, CT, MR, etc) using some form of x-ray source.  The results might lead to a diagnosis or the issue might be elevated to a specialist.  If the issue requires surgery, one would be referred to a surgeon.  It is clear that just being associated with a healthcare organization does not qualify an individual to perform brain surgery.  The same holds true for software development.  I know of a receptionist that used to make minimal wage at a software development company.  One day I was her reading some “For Dummies” book.  A month later she got a raise that increased her hourly rate by a factor of five (5).  She was now a software developer.

If you watch some television you must have seen a commercial for a prescription drug used to treat / prevent acid reflux.  I watch (typically sleep in front of the television screen) television about an hour a day.  Around 08:00 PM, based on workload, I tend to start shutting down my computers.  I typically run three (3) different computers.  Given the resources available on the different computers and their intended uses, it is not technically or economically feasible to virtualize the dozen machines that I have at home.  That said I do run VirtualBox on a couple machines mostly to access a few VPNs (Virtual Private Networks) that have side effects on the network and to run different operating systems (Linux and different flavors of Windows) on the same hardware.   Getting back to watching television, it was not until a few months ago that for the first time my wife and I, at insistence from one of my sons, that we got DVR from the cable company.  Now we record a few programs during the day that we might watch when we are available to spend an hour or so in from of the flat screen.  No matter what my wife and I have going on during the day, on workdays around 08:30 PM, we watch World News with Diane Sawyer on ABC News.  If you watch television around 05:30 PM, it becomes obvious the demographics watching based on the type and amount of commercials.  In particular there is one that start with an actor wearing a white physicians coat attempting to use a jack hammer on what appears to be a road construct ruction zone in a downtown area.  The person does not fit in the scene.  In the next scene a husky male, wearing blue jeans, boots and a tool belt is wondering in what appears to be a pharmacy aisle looking for medication to cure the symptoms that he appears to portray (heartburn / acid reflux).  In a following scene the physician is discussing with the patient the symptoms, which lead to a prescription for the pharmaceutical in question.  The moral of the story / commercial “you do your job and let the physician do his” or something among those lines.  Perhaps I should get a lawyer and demand some type of royalty from the AtraZeneca Nexium commercial.

One can easily draw an analogy between technicians, engineers, system managers, software developers, computer scientists, QA, chief technical officers, chief information officers, IT managers, and many other people whose job is associated with computers and / or software at different levels (users, super users, system administrators, maintenance, developers, analysts, scientists, etc) whose jobs at different organizations are associated with providing computer and / or software based services / products.  Such organizations tend to have receptionists, clerks, supervisors, managers, etc, etc, etc.  An individual just by belonging to such organizations do not qualify to design software.  There is of course the exception when the person reads a book “For Dummies” and in a couple weeks is developing software.

Requirements are extremely important.  They should unambiguously document the use of some software to be / being developed.  The idea is quite simple.  If the intended use of a software product cannot be unambiguously documented then no one knows what to do.  If no one knows what to do there is no way to generate a design or a set of test procedures to test the software because no one knows what the software is supposed to do.  This simple logic is seldom put to work.  Most software projects are started and in many cases completed or canceled before a useful requirements document are generated.  Different methodologies have different ways of generating and maintaining requirements.  Most methodologies are flawed.  Some make sense but are not well known or their steps are not easy to follow.  One may be led to believe that there is no good software development methodology.  This is far from being true.  As an analogy one can take diets.  Not sure how many diets are available.  Some seem to work for some for some periods of time.  Most fail miserably.  The fact is quite simple.  Our bodies are engines.  Engines need fuel.  If our bodies burn 1,500 calories per day and we intake 2,000 we will just be gaining weight.  On the contrary, if we change our life style by incorporating exercise and reducing our caloric intake, then if we consume 3,000 calories per day but due to exercise and other healthy habits burn 3,000 calories in the same time period, we might not live one hundred years, but on average we will have a longer and healthier life.  Like I said, there are many ways to skin a cat but there is only one way which is best.  Burn the same amount of calories as one consumes on a daily base and forget any other type of diet.  In reality software development is several orders of magnitude more complex so things are not so simple as a diet.  What is extremely important is to have unambiguously defines requirements written in a succinct way.  Writing a Victorian novel is not the proper way to write software requirements.  The detail and number tends to be a function on the level of expertise of the lead system architect and the development team as a whole.  Being involved in many software development projects and having knowledge of different methodologies attain this level of expertise.  As we all know, no matter how the metaphor is applied, there is never such a thing as a free lunch.

Design by committee is a term that is typically used for both design and requirements even though these are two separate aspects (requirements and design) of any software development project.  A good designer needs education and experience.  A person without a software background is able to design basic software.  Today wizards allow individuals with little or no computer science education to create software (specially when it is a GUI web based) not understanding or knowing what they are doing.  Such approaches tend to work now and then form small companies.  As the demands for the software grow, there are no wizards that can help.  One needs to understand what is going on and be able to make the necessary changes or come up with an alternate solution.  While developing software one runs into situations when the desired effect is not achieved with the initial approach.  One then needs to look for alternate ways to achieve the goal.  This process is based on know how and experience.  In cases when one needs assistance, one should be able to look for it consulting with other team members or vendors.  In many cases, the higher the position in a software development project, the least the person is willing to accept that help is needed.  When too much involvement by the entire team on all aspects of the project is reached, no one is responsible for failures.  Humans tend to blame others.  When the software is properly partitioned and individuals are assigned to specific modules, developers tend to take responsibility for their part.  This is especially true when a separate group does testing.  That way the team as a whole responds to issue encountered by QA.  If one looks at standards that have been developed by many individuals from different companies (e.g., DICOM), the results tend not to be elegant, bloated and the software implementations are poor (slow and consume lots of resources).  When single individuals come up with something (e.g., Linux) the result tends to be elegant and performs well.  This does not mean that a single individual developed Linux or that all members that participated in the committees that developed DICOM contributed.  The rule of thumb is to assign parts or a project to a single individual and develop the system in such a way that all interactions are based on clients making requests and a server responding to them.  This can actually be done with client / server interactions or via the use of APIs.  The key is to make each module responsible for their operation based on the requirements for the software.  Once again, being able to partition a project of any size requires know how and experience.

Lead by example is something that is not easy to do and even harder to follow, but when achieved, results are typically success.  The idea is for the senior / lead system architect to show how things are done.  It all starts with requirements and their associated documentation, through design with associated documentation, implementation and associated unit testing to product delivery and if needed initial support.  Less experienced engineers, when properly motivated and interested in learning and advancing tend to imitate success.  As usual this is easier said than done.  If one has a single rather simple project under the belt and the software is supported by an army of technical service personnel, then that might not be the person to put in charge.  The same holds true for people that work as system administrators and are asked to design software with little education and experience.  Being able to issue database queries and use a report writer to generate reports with colorful graphics does not qualify the person to become a lead system architect.  Once again, politics get in the way of reason and after a few years when the software is a failure people stops and thinks on what was done.  At that time it is too late for the software and the project.  It is better to address most (all is not possible) situations before they turn into issues and then into problems.

A couple decades ago I was involved in the development of the first document management system ever.  The project started by processing documents for insurance companies and then shifted to address medical images.  The general manager had a PhD degree in nuclear physics.  He used to work for a well-known company (Xerox), which at the time had a quite known research center in Palo Alto, California.  On paper this individual fit the bill to manage the project.  This individual had quite an amenable personality.  He might have been top notch in nuclear physics but had little understanding (to say the most) on hardware and software development.  At the time, there was no off-the-shelf standard hardware or software to put together the product.  Top executives at the fortune 500 company backed up the project.  The first and probably largest mistake was assigning the lead system architect role to an individual from IT.  He might have had experience with databases and report writing, but that was it.  The lead architect had to experience or knowledge on hardware, networks, imaging, etc, etc, etc.  As the project blew out its schedule, the perceived solution was to throw at it more money.  With that extra funding additional resources were brought in.  The project lasted several years.  Towards the end, software QA was a 24 x 7 operation.  Each day new issues were discovered.  At some point in time after over 100M dollars (in 1980’s value) the original project was cancelled.  Somehow after a few months a second pass for the project, under the same guidance was authorized.  The system architect blamed the computer language used to program.  The first pass used C so the second was done in Pascal.  I left when the first project was cancelled.  After a few more years (do not know the actual cost) the project failed and it was cancelled for a second time.  The lead system architect was able to get authorization with very few resources to run a final and third pass.  This time the software was written in Ada (do you know what Ada is?).  Of course, the language has nothing to do with the success or failure of a project.  After a year or so the project was finally put to rest permanently.  What is interesting to note is that the lead system architect returned to IT to deal with databases and report writers.  I ran into him at a seminar in a hotel in downtown Minneapolis.  He was doing just fine back were he started a decade or so before.

Testing is important in any software development project.  The same individuals developing the software cannot do testing.  Developers must create the methods to perform unit testing.  System testing must be performed by a different organization.  Such tests must be developed following the software requirements.  QA should never see the design documentation nor be exposed to the source code of the project.  The issue is that testing then becomes a function of the actual code and not of the requirements.

For software to be successful it must be flexible.  If the software works perfect for only the specified set of test input and it fails under different situations, the quality is not there.  The software must be designed in a way that it can evolve not breaking previous uses and allowing for foreseeable changes in the business.  Very few software modules are able to adapt to changes and survive a decade or two.  When the software is properly designed and implemented with simple yet robust technology, the software product tends to be useful for long periods of time.  When little though is put into a project, the resulting software tends to become obsolete in as little as a year or two.  Some companies thrive in such models because they imply recurrent business.  If you do not understand this, take a look at the different technologies provided by Microsoft.  Most of them do not last more that a couple years.  Every time they are replaced with something quite different with the promise to solve all problems.  This is type of behavior / approach is known as the “silver bullet”.

Team organization is quite important for any software development effort.  This is true from projects with a single person to ones with hundreds of people in multiple disciplines, which may interface with products / modules form other companies.  Putting together a team is important in order to have a successful project and quality software result.  Tem members should be selected based on expertise and track record.  Getting back to the healthcare analogy, you would not want to have surgery performed on you by a receptionist just because you like him / her.  You can always have some simple surgical procedure done by a general surgeon, but if a specialized procedure were required, only a specialist would be considered.  Software is no different, yet people tend to go against basic logic and allow people with limited qualifications to take charge.

Politics in projects, no mater the size of the organization or the team (at least two people are needed to disagree but a single person with his / her own agenda qualifies) needs to be addressed and managed for a project to succeed.  This subject appears to be foreign to most people, yet many books have been written about the topic.  I have personally read a few.  Software development is a human activity.  We as humans tend to resist change and have preconceived ideas as to what needs to be done.  This is just one reason for having a simple, short and complete set of requirements for a software project.  By no means this statement patronizes the waterfall model.  There are different methodologies better suited to collect and maintain requirements during a software development project.  A properly managed project assigns team members to different tasks / modules based on their education level, experience and track record.  I have run across several individuals with advanced degrees (PhD) that could not develop software even if their life depended on it.  The issue is that humans tend to be polite and not tell individuals that they should be doing something else or they just tell other members their thoughts but no action is taken.  This builds friction which when paired with lack of expertise, produces an environment in which success is typically not achieved.  The best way to prevent political problems is to put together the proper team and assign tasks that are palatable to the individuals.  By having quality requirements and proper architectures, team members can see how their contributions to the software make a difference.  If a problem is encountered, developers are more inclined to look at their pieces and correct them when needed, than just blaming other modules (pointing fingers).  Some times developers go the additional mile by making the necessary changes to their modules to compensate for the shortcomings of others while still keeping the integrity of the software.

After reading most of the contents of this blog entry you might come to the conclusion that the best software development team is a completely homogeneous group.  This is not the case.  Diversity is important.  The issue is to make sure that diversity comes from backgrounds that may contribute to the project and that such individuals are not charged to lead.  Different ideas should always we considered but when all is said and done, someone needs to make a decision as to how the group should proceed.  During my professional career I have had the opportunity to work for small, medium and large size companies.  I have also done contracting / consulting work.  My background is a mix of electrical engineering and mostly computer science.  I have worked in quality assurance groups and have supported systems at customer sites.  That diverse background has helped me come up with innovative approaches to solve design problems.  Some time ago I was working for a company in the east coast.  I was in charge of developing a color console for a process control system.  The company was developing the entire system.  The system could control the environment in buildings or an oil refinery.  The product was software and hardware based.  My team had a PhD in psychology.  Obviously the individual was intelligent.  He was also very dedicated.  Married without children, just a cat at home.  Because of this, we were able to work long days and enjoy them.  In a few occasions this team member would get stuck in a task.  Without technical assistance he would spend several days in a rut.  I recall one occasion when we were dealing with a driver for a UART.  Given that his background was not in computer science or electrical engineering, the concept of how to control the chip to process RS232 input and output in queues at the interrupt level was overwhelming.  After I notice that we were not moving forward, we got together and in a day, with a different design, the issue was addressed and we were able to continue to make very good overall progress.  A year or so ago I was optimizing a file transfer for an application on Windows.  The specific transfer operation was not intended for what it was designed.  I spent a few days optimizing the command line interface using a similar approach as to the RS232 driver I worked with my team member a couple decades ago.  Of course the issue was over the TCP/IP network and had multiple data buffers and queues and was working with multi core processors.  The point is that knowledge and experience were brought up to solve two separate but similar problems decades apart.  Diversity in backgrounds is most of the time beneficial specially when used the proper way.

There is a cryptography expert that has made his home the Twin Cities of Minneapolis and St. Paul.  I have read a few of his books and periodically skim over articles in his newsletters.  He has a friend / co-worker that has repeatedly made the comment (perhaps is more than a comment but a believe) that the software development industry is in shambles.  This person claims that all software is full of bugs, insecure and not stable (and he is not just referring to the Windows platform).  I have read some of his articles and based on my experience have to somewhat agree with his opinions.  The matter of fact is that software, no matter which company is producing it, has many issues that can be traced down to all of the development phases.  In general properly trained individuals with proper experience do not develop software or the team members and management have different goals.  In most cases the goal is just to get a promotion and raise and move or to a higher position in the same or different company.  Company managers in most cases are not interested in the quality of the product but on making money.  This behavior is so widespread in our country that we have become oblivious to it.  A few months ago the biography of Steve Jobs was published.  When it became available, given the current set of books sitting on my nightstand, study desk and other places I do not wish to mention but most people should be able to infer, I decide to skip.  My wife got the book for her Kindle.  She already finished reading it.  Several times a day I would ask about what she was reading or she would just volunteer comments.  I feel like I read the actual book.  Steve Jobs was not an easygoing person.  He had several traits that I just dislike.  On the other side of the token, Steve was a very passionate person as far as the Apple products and services.  He always set his goals to achieve perfection.  Although that was not possible, Apple products are far superior to Microsoft’s.  This is quite obvious at the Mall of America in Bloomington, MN.  On the first floor the Apple store is just across the Microsoft counterpart.  I do not visit the Mall every day, but when I do it calls my attention that the Apple store typically has at least double the amount of customers.  This is how typically taste tests are conducted.  The subject is presented two (2) options and based on statistics, the best tasting food is the one more often selected.  Without a doubt the winner is Apple.  Good job Steve!!!

How to collect and validate requirements for a software product is just one more important part in the software development process.  The same holds true with design, implementation, testing, documentation, etc, etc, etc.  The design aspects, not covered in this blog entry, describe how the software product is designed to cover all the requirements.  Basic design concepts and some techniques used in cloud computing will be described in a future blog.

If you have a comment / question please feel free to leave a message.  I will respond and post it on this website.

The Naïve American

Virtualization, as far as computer technology goes, is currently a hot but not a new topic.  Virtualization was developed in the 1960’s by IBM to allow very expensive main frames to be shared by multiple users.  The reason was simply based on cost.  At the time, it took in the order of several minutes to a few hours to set up programs and associated data.  Today, on all modern computers, we are able to invoke programs with the click of a mouse or by typing in a second or two a command on a console.  Depending on the intended use of the computer, we are able to utilize our personal computers in bursts that may hit close to 100% while remaining most of the time almost idle 0% to 1%.  On Windows you can see this by opening Windows Task Manager and selecting the Performance tab.  As I am writing this blob using MS Word, I can see a few CPU utilization spikes per minute in the order of 3% to 5%.  The rest of the time it is 0% to 1%.  What a waste!  Of course, when I am working developing software or using other more demanding applications than just a word processor, the CPU utilization may be at 100%.  This frequently occurs when I am compiling a program or running tests on software that I am developing.

How could I possibly benefit from using virtualization?  The answer at this point appears to be just to use technology for the sake of technology.  But wait a second.  I do run some applications that need a different operating system and have some applications that make use of some software that interferes with my network functionality when I use it.  By default my machine boots on Windows 7 Professional, but I often have the need to run different flavors of the Linux operating system.  As in the 1960’s, I could serialize my requirements by booting Windows to run Word and Visio while working on documentation.  I would then close the applications and shutdown the operating system.  I could then boot with Ubuntu Linux and do some software development.  By switching back and forth I could meet my needs, but if I run virtualization software I could simultaneously be running Linux and Windows on my machine or properly speaking, on two (2) virtual machines.  I could then switch from one to another with a click on the mouse.  That makes sense!

A couple weeks ago I attended a one-day Microsoft seminar in a hotel in Bloomington, Minnesota.  The location was easy to get to.  The hotel is next to I-494 and France Avenue.  Food and beverages were provided.  The venue was as good as expected when attending a Microsoft sponsored event.

Overall I enjoyed and learned about Microsoft offerings for virtualization.  The presenters had the sense not to trash competitor’s products.  Overall the general consensus was that most offerings provide similar value and are priced somewhat different.  Of course, if you are considering virtualization and your organization is a Microsoft shop, it does make sense to consider Microsoft offerings for virtualization.  In general products from the same vendor tend to play well among each other.

For the most part, attendees appeared to be system administrators.  Some showed various knowledge levels on the presented products.  A few people pick up their belongings during the presentation and headed out.  They did not seem interested with what was presented or the level and depth of the seminar.  I stayed for the morning session and left before picking up lunch box.  The presentations were fast paced and focused on product features and how simple (or complex) it is to get into and use virtualization.

During the first break, I approached one of the presenters and ask why should one use virtualization in the first place.  Apparently this is not a typical question asked by the average attendee.  I am relatively familiar with the concepts and use VM VirtualBox for work and research.  For my simple needs any virtualization tool is quite adequate.  The presenter appeared to be startled at my questions so I continue to fill in by stating that if the applications of interest run on Windows why should I use virtualization at all.  The answer was quite unexpected.  It went something like not all Windows products play well with each other.  For example, if you are using SQL Server you probably do not want to have in the same machine Exchange because those products do not play well with each other.  My initial comments were that virtualization is a way to hide limitations and issues with Microsoft products.  At that point the presenter kind of gave me a nod with his head like if he had successfully answered my question and moved on to listen to a different question specific to a virtualization products from Microsoft.  It was obvious to me that virtualization as used by most system administrators and for what I saw and learned during the presentation, is just a technology pushed by Microsoft to technicians and system administrators that do not appear to understand the concepts and reasons why one would virtualize.

During the next segments it became obvious the overhead incurred by virtualizing a machine and the additional issues that are present when different machines have different and variable loads.  Memory assigned to a virtual machine can cause it to start paging / swapping even though there might be additional free physical memory in the host computer.

In general, hardware is optimized for use by a computer system.  When an array of virtual machines, each with different requirements are hosted on a computer, the performance of the processors / cores / memory caches are affected.  It is true that hardware instructions have been added to the repertoire of processors to help with virtualization, but they work fine when a single virtual machine is used.  The only valid justification for virtualization is to run disparate operating systems on a single computer.  This reduces the amount of space and energy when compared with discrete computers needed to run disparate operating systems.

When you get a change, open Task Manager on your Windows computer, and without running any applications, take a look at the CPU utilization.  Make sure to show kernel times.  Depending on the system and network connections, one may see from around 1% to 5% CPU utilization.  What you see is the operating system overhead.  Utilization grows as applications run.  The additional overhead is due to operating system tasks that are periodically performed.  Notwithstanding the additional overhead when the system gets some basic load, if ten (10) machines are virtualized on a computer, then each virtual machine will be using 1% to 5% of CPU when idle.  If one takes the sum, the hosting hardware will be using about 10% to 50% of the CPU resources supporting idle operating system tasks.  It does not take much to figure out that having compatible applications / servers running on their own sand boxes just to make use of virtualization does not make sense at all.  The software applications / server running directly on a single instance of the OS would have available a considerable amount of resources that are wasted by duplicating OS tasks on each instance of a virtual machine.

By the way, stop for a second and take a look under the CPU Usage for Memory utilization in the Task Manager windows at the amount of memory that each virtual OS is using just by having it running idle.  A Windows 7 system takes over 1 GB of RAM when idle.  In the above example, ten (10) virtual machines will waste 10 to 15 GBs of RAM.  Of course, who cares when most servers hosting virtual machines are configured with 128 or more GBs of random access memory.

Target is big in the Twin Cities area.  The main example of a company that has ripped the benefits of Microsoft virtualization was what Target has done for their stores.  The project to simplify operations and reduce cost was to replace seven (7) computers with two (2) each running seven (7) virtual machines.  The contents of each virtual machine were not covered during the presentation.  Of course, if different software packages requiring different flavors and versions of operating systems were required, then virtualization made sense.  The two (2) physical machines are running a cluster so if one fails the other picks up the entire load.  One has to assume that in the Target example, some virtual machines were running different operating systems (i.e., Linux, Windows) while others were running different Microsoft servers that do not play well when in the same machine.

In conclusion, the use of virtualization just for the sake of virtualization is a performance hindrance to the entire system.  Poor architecture, design and implementation can be hidden to some extent with the use of virtualization, but this comes at the expense of powerful servers when the proper develo0ped software could run directly on a single non-virtual machine with considerably less resources.

One of the main reasons for the proliferation of virtual machines is that most technicians making IT decisions do not have the background or experience needed to evaluate technology and select it only when it provides a benefit to the organization.  Also, you have vendors whose business thrive in making periodic changes and motivating system administrators to keep up with the latest technology just for the sake of it.

The Naïve American

At a high and introductory level, the book provides some good definitions that further open the appetite to pursue more in depth reading on most topics.  Being a system architect, software developer and business owner for over twenty years, and the book did not provide much food for thought.

On several sections, it appears that what is stated on a paragraph might be contradicted on other.  No good rules of thumb are provided.  Perhaps if the book is rewritten to cover better topics in more detail, the book could be useful.

The authors state that predictions, especially in computers and software, have in most (not to say all but the most obvious and simple ones) have proven wrong.  Nevertheless the authors go and make ten predictions.  Some of them might in the future become a reality, but some seem to be quite incredible.  I would suggest for the authors to have followed their own statements and skip predictions.

The book has several pictures and photos that may or may not be what they are touted to be.  Most descriptions of how Google, Facebook and other Cloud vendors operate are based on assumptions.  While reading some of their descriptions about some private data center, the thought occurred to me that I was at a supermarket checkout reading tabloid headlines.  Several sections in the book sounded like a white paper by a vendor trying to convince the reader that if they do not use their product they will never be successful in their business pursuits.

I would not recommend this book.   Hopefully the authors have other good titles which I have not read yet.

The Naïve American

 A segment titled “Jack Abramoff: The lobbyist’s playbook” showed to Naïve Americas something that has, is and will continue at the highest levels of politics in our country.  When you are not a politician you might be faced by business situations in which you are privy to some information that could make you money called “insiders trading”.  For example, if a company is about to purchase a second company, making the stock of the second company be worth much more that its current value, and based on this information which is only known to top executives, you go and buy stock on the second company, you have the opportunity to make a nice profit in a matter of hours or days.  The laws of the United States of America forbid such behavior to typical Naïve Americans.  As a deterrent such actions are punishable by several years in prison.  BUT if you are a politician passing laws that will make a company’s stock go up in price and based on such information purchase stock and after the law is passed, sell it at a profit (insiders trading), you do not have to worry.  The law, made by and for politicians, does not consider your unethical and immoral actions as a crime.

During the 60- Minutes segment, Democratic and Republic politicians were asked if they had profit from “insiders trading”.   Most of them denied it.  Some acknowledge it by stating that for them it is not unlawful.  This is why 99% of politicians leave office millions of dollars richer that when they were elected.

About a decade ago, I was looking into doing business with countries in Latin America.  One of the forms I obtained from the US government to consider expenses for tax purposes had an entry labeled:   Bribes.  It is considered common business practice in Latin America to bribe politicians as part of regular business.  Of course, in the USA no one would ever consider bribing politicians.  We have laws that make legal all types of bribes placing our politicians on the top of corruption.  The only difference is that we make it legal.

I know of a couple brothers in the Twin Cities area that paid off politicians to move the location of Interstate 35 in order to have it pass by their property considerably raising its value.  During the 60 Minutes segment, it was mentioned, a politician that knowing that a road was going to be built, purchased land at discounted prices and after he made the new road pass by his new property, he cashed in a nice profit.

Not sure if the 60 Minutes segment brought up something new to most Naïve Americans, but politicians have been taking advantage of their position since the USA became an independent country.  With time, politicians have made huge advances in power allowing them to make millions of dollars in office while many Naïve Americans part of their constituency have, are and will be loosing their jobs.  I guess we called this the American way of life.

Next time you are going to the polls, make sure that the politicians you are electing are less prone to corruption.  Try to push current politicians to change the law making it a crime to take advantage of their political status.

The Naïve American

The book deals with a conspiracy in London in the 1700’s.  The book is quite engaging.  I could not put the book down.  Read the thirty-six chapters (about 440 pages) in a couple weeks.  Given that I work full time and spend reading technical books for a few hours a day, for my standards, this book was quite interesting and kept me glued to it.

The plot has to do with several murders, stocks, and ethnic backgrounds.  The book is based on the first stock bubble in history.  The book was the first of a series written by David Liss.  It received a New York Times Notable Book award.  I highly recommend it.

I read about three non-technical books a year.  I did enjoy this one.

The Naïve American

The first dealt with the amount of savings that would be made (in the order of billions of Dollars) if our government decides to replace the use of Dollar bills with Dollar coins.  The savings are based on the replacements costs of the One Dollar bills.  As far as consumers is concerned, there is little, if any, impact on carrying one Dollar bills or one-dollar coins.  I am not a young pup so my habits may differ from the younger generations; nevertheless, I typically use a credit card for most (with few exceptions like the occasional lottery ticket which must be purchased using cash) of my purchases.  My wife and I try to keep the card with a zero balance so we pay it in full each month.  A nice feature of using a card for purchases is that at the end of the month you get an itemized list of purchases.  If I use such approach, I would assume that most Naïve Americans would be fine with using coins instead of bills.

So why the switch is not a reality?  The issue as usual is special interests.  One has to consider the companies involved in providing the materials (i.e., ink, paper, plates, print press, etc) for the One Dollar bills.  We also have to consider vending machines.  Most are configured to accept bills.  Few are ready to accept one Dollar coins.  As usual, special interests pay off politicians and who cares about then Naïve Americans that got them elected in the first place.

The second story relates to wealth distribution in our country.  Statistics show that the top 1% of the population has grown richer by a factor of three while the remaining 99% has become poorer.  Something is not working at all in the USA.  Naïve Americans claim that we are free and live in the land of opportunity.  That may be correct for a very reduced minority.  The majority of Naïve Americans are experiencing tough economic times.  On one side one we have Republicans, which do not want to raise taxes because they will reduce jobs.  Not sure who came up with that twisted logic but it appears that millions of Naïve Americans believe it.  On the opposite corner we have Democrats who wish to raise taxes to pay off unions so they can be elected to office.  What about the majority of Naïve Americans?  Most of us do not benefit with Republican or Democratic governments.  We need to elect a president that is not Democratic, nor Republican but American.  Of course such candidate does not exist.

At some point in time in the very near future, the top 1% of the population, which are the ones that control politicians, need to realize that some changes are needed if we wish to preserve our way of life.  When I say our way of life, I refer to the economic climate of the early 1950’s.

Hopefully a few Naïve Americans in the top 1% will come to realize the gravity of not inducing a change and will put some money and effort in helping our country.

The Naïve American

As usual, when I am in the process of starting work on a new project that uses a set of techniques of which due to my work experience (I am not a young pup) tend to be quite fluent and comfortable with most of them, I purchase and read a few books in order to learn what I do not know or I have forgotten because I know about a topic / technique but have not used it in years.  Simply SQL is a very good example of a book that could easily take someone that is a novice in SQL to an individual with a good base on where to start tackling real world situations encountered by programmers and database developers.  As we all know, one can read all the books available in any particular subject, but without practice and experience to solidify the concepts, it is almost impossible to become proficient.  Would you trust a physician that has read all the necessary books in medical curricula for a specialty but has never practiced or seen a patient?  Not sure about you, but I certainly would not.

The book is a good introduction and refresher for SQL.  Plan to keep it handy to replace my old “SQL Pocket Guide” by Jonathan Gennick published by O’Reilly in 2004.  Have used the pocket guide since 2005, but due to Google and the Internet, when I need syntax I just get it on-line (much faster and no need to carry the pocket guide).  The Simply SQL book appears to be much better suited to be used to start writing more complex queries from scratch.  Only time will tell if this approach is valuable.

In general I was quite please reading “Simply SQL”.  One might consider consulting it before starting to write a complex query.  I recommend for developers who casually use the services of a SQL relational database.  The book appears to be too simple for database administrators and developers who spend their day using SQL.

The Naïve American

PS:  The Naive American corrected / enhanced the information in this post after receiving a message from Rudy Limeback.  Thanks for setting the facts straight and for authoring the book!

Yesterday was one of those days in which driving made sense.  Drop the dogs at the sitter around 10:00 AM and headed to Albertville.  The first store we visited is Polo Ralph Loren.  For the first time we found boxes with six white t-shirts.  Picked up a couple.  We added some other items and got in line at the cashier.  While our items were being scanned, the attendant made the comment that the 30% discount would be applied to all items after ringing them.  A lady at the cashier next to us turned around and handed us a coupon she had printed on her computer.  The page has a Polo Store additional 20% discount on all items.  The coupon was only valid on purchases totaling over $100.00.  Her total just made it up to $80.00 so she was kind enough to pass it on.  We handed the coupon and the attendant obliged by giving us an additional 20% off.  That was great, we picked up items marked down 30% off and we gout an additional 20% off.  My wife and I were happy to receive 30% + 20% == 50% off.  We paid and moved on to the next store.

I decided to walk to the car and drop the bags.  On my way back to join my wife I pulled out from my pocket the receipt and briefly checked the numbers.  Given that the first item was listed for $49.99 it was obvious that the discounts would be as if the prices was $50.00.  In other words the discounts would be $15.00 for the 30% and $10.00 of the additional 20%.  That was NOT the case.  The second discount was $7.00 (not $10.00).  Obviously going back to the store to discuss this with the clerk or store manager would be a complete waste of time.  I am willing to bet that they are not the ones that program the point of sales machines.  The discount comes from the actual owners of the store or possibly from the corporate headquarters which set pricing all over the country.

I am sending a link of this post to the Polo Store to see if I get some feedback.  Chances are that somewhere in the store there is a sign or brochure with some very small (almost illegible) font size disclaimer that actually states (justifies) the policy (scam).  Just the need of having a disclaimer is an indication that what is advertise is most of the times some form of deception.  This appears to be the way business is conducted by most companies in our country.

There is a furniture store a few miles away from the Albertville outlet.  They have periodic sales.  The ads on TV state that all items are reduced by 50%.  A disclaimer then starts listing the most desirable brands that are not included.  If one watches the TV commercial not paying attention to the fine print and make your way to one of the few stores in the area, then given the time spent in getting to the store and looking for the item(s) of interest might convince you that given the investment of time and fuel that a lesser discount is acceptable.  This is a technique used by professional negotiators.  In other words, many companies use their resources (lawyers and marketers) to take advantage of Naïve Americans.

The advice from the Naïve American, before you spend time looking at something that is too good to be true, make sure you read the fine print in the ads and drop the idea of purchasing an item that appears to be very discounted.  That way the amount of time you have invested is minimal and you feel comfortable walking away from the deal.

The Naïve American

Besides the general treatment on how to negotiate which according to the author is based on: intelligence, power and time two (2) things mentioned were quite interesting and close to me.  The first relates to the fact that things do not just happen because you know you have the truth and knowledge to backup your position.  I have learned through life that this a very important point to always consider.  I personally have a hard time accepting this due to cultural and family values.

The second item is that of power.  You can negotiate all you want with a person that has no saying in the matter and the results will always be the same.  Nothing would change because the individual at the other end, even though it might sympathize with your cause, has no way to affect the outcome.  I have seen this in action many times when a person (who understand the power issue) asks who is in charge before saying anything else because it is just a waste of time.

Last but not least time.  This is very easy to see and understand.  It has happened several times to me.  You go to a store and engage a sales person with questions.  As time goes by and you get answered most or all of your questions, yet you remain hesitant as to purchase the item.  Silence seems to work at your advantage most of the time.  You might even leave the store and come back asking to continue the conversation (more unanswered questions) with the same salesperson.  The sales person has invested so much time with you or might be tired of repeating the same things that he / she will figure how to close the deal by offering additional discounts or other creative ways to finance the item in order not to call the experience a total waste of time.

The Naive American highly recommend this book.  It is interesting, keeps you captive while reading, it is relative short and inexpensive to purchase.

About a month ago the State of Minnesota run out of money.  After several weeks, a compromise was achieved and a new budget was approved.  Most Naïve Americans did not think much of it, at least that was what people interviewed on TV was voicing.  The Naïve American saw the entire exercise through a different colored glass.  Allow me to explain.

The issue was a disagreement on the new budget.  All families in this and any other country tend to live within their means / budget.  On occasions when for some reason or another they do not, the family has to file form bankruptcy and are forced to change their life style.  In America it appears that that rule should not apply.  Why you may ask?  Well because we live in a crystal tower up in the sky.  In other words, there is no reason for common sense rules not to apply to all of us.  When the State of Minnesota does not have enough money for their next budget two (2) things have to happen:  reduce spending and increase taxes.  The cuts appear to be quite simple.  We have so many social programs that spend so much money and the returns are dismal.  On the other hand, most large companies avoid paying taxes because of laws and exceptions they have obtained by “bribing” politicians.   Bribing means legally providing money and deals (i.e., books, future consulting jobs, etc) to get them elected and to hold their lucrative positions.  It is not only the companies that bribe politicians; unions do the exact same thing.  Mr. Dayton, the current governor of Minnesota, was told by the union of school teachers that they elected him because of the promises he made to them to continue to raise indiscriminately their pay and benefits even if the results are not there.  By the way, results do not mean comparing students within the country but rankings should always include students from around the world (including China).  Yes, Minnesota students rank better than students from Alabama, Mississippi and New Mexico, but how against other countries which in the eyes of Naïve Americans have lower standards of living than the USA?  You can figure that on your own.

The governor (a Democrat) and the members of the house (mostly Republicans) were elected by registered voters that live in Minnesota with the idea that they will spend their time and efforts to improve the lives of people in the state.  As the budget deadline approached, the number of CLOSED DOOR MEETINGS increased.  What in hell can / should elected politicians discuss that the people that elected them should no hear about?  The answer is NOTHING, yet they did and to top it off they missed the budget deadline.  Because of that, millions of dollars in revenues were lost every single day while the government was shut down.  Naïve Americans (not including me, but my grand daughters) that enjoy state parks in Minnesota had to cancel their vacation plans because the parks were closed.  How is it possible for Naïve Americans to put up with such behavior?  We believe we have so many rights and live in the land of the free, apple pie and blue sky.  In actability a reduced number of incompetents career politicians elected for and by selfish people have more saying that they should be allowed.

In conclusion, Naïve Americans living in Minnesota should write don so they do not forget the names of the politicians involved in the state budget fiasco and send them a copy of the “You Can Negotiate Anything” book and by the end of their current terms, make sure no one ever votes again for them.  We do not need people like the governor that after the shutdown decided to take the weekend off and let laid off people wait because the elected official wanted to enjoy a summer weekend (

The Naïve American

PS:           Scott, when done please return the book.