Did you feel that?

The Cloud will be Adopted just as Fast... er... Slow as the LAN

Jim Wilt — Mon, 29 Jun 2009 17:45:00 GMT

Riding on the shoulders of giants: This assembly of thoughts is inspired by the great works of others including Above the Clouds: A Berkeley View of Cloud Computing, Pat Helland, Miha Kralj, Pallaw Sharma, and many other industry experts & references such as Gartner, Forrester, etc.

First, let's take a short sidebar to explain what the Cloud is: I'm going with those who describe it as Elastic Computing as follows. If I have a data center, I must have sufficient infrastructure so that at maximum utilization, I am only using 85% of my IT hardware resources lest I crash everything. I do this even though my actual utilization fluctuates over time:

When doing this, I am paying for the infrastructure 24x7 which includes power, cooling, real estate, etc. Let’s say, for illustration purposes, that I run an efficient shop and can do this at a rate of $1/CPU-hour. If my data center supplies 1000 CPU-hours per day, I’m going to be paying $1000 a day for my self–supplied hardware resources. If for some reason I spike over my 85% threshold, I’m toast.

Placing select IT services in the Cloud means that I will get elastic resources so that I only pay for what I use, with no upper limit (a sudden spike is absorbed by the Cloud). Now, my cloud vendor is going to charge more per CPU-hour, say $2.56/CPU-hour - ratio based on current Amazon EC2 costs, than what it costs for me to host this myself, but because I only pay for what I use, I can realize a net savings:

So, if I only use 300 CPU-hours a day, my costs are $768 ($232 less). If I spike, I simply pay more for the additional capacity I need, no limits (in theory; in practice I may see a performance hit, but no crashing).

Those are the easy numbers based on published research around elastic computing, but there’s more to the numbers than just this.

Next sidebar, who are the cloud providers? As of this writing, Amazon EC2 is at one end of the spectrum and Google Apps & Sales Force at the other. In between resides Microsoft's Azure. Each has its own model around consumption and development in their Cloud. Miha Kralj has effectively illustrated this as follows:

On the left, you have fewer constraints, however, to reap the benefits of elasticity, you must implement it yourself. On the right, the elasticity is virtually automatic, however, you must develop using specific tools under specific guidance.

Final sidebar, who are going to be cloud consumers? Contrary to what cloud providers might say, my personal experience over the past 4 years suggests the following:

Why as slow as the LAN and not as fast as the Internet? The internet was easy. It was as intuitive as reading a magazine, only more fun because you simply click on a link and poof, you’re somewhere else. No real training, and while there was a tremendous infrastructure behind it, to the end users, it was just there and it worked.

While the LAN was also a major disruptive technology to business, it rode on the coattails of PC adoption which also had its hurdles. Corporate IT departments at best thought of PCs as glorified 3270 terminals. Pop in an IRMA card and poof, your PC is now part of the corporate network! Next came file and printer sharing on ARCnet and later, Ethernet networks. At this point, momentum began to increase. Major corporate data repositories began moving to LAN file servers as accountants liked their forecasting spreadsheets running from Lotus 1-2-3 and could justify the move by the value their immediate forecasts provided over waiting for mainframe reports. Note, in most cases, individual end-users inside corporations pushed business to reactively adopt these migrations.

It wasn't until LANs became mainstream for smaller businesses and home networks that it fully took off. That’s when real peer-to-peer applications took the LAN to the next level. What were the first peer-to-peer applications? Games and instant messaging followed by share database systems.

It took business 15-20 years to adopt technology to the same level that individuals readily adopted in the the form of the Internet in 2 years.

Because the Cloud is about how things operate behind the user interface, its visibility to the general public will be very much obscured. As identified above, public individuals adopt faster than business and their influence on business often accelerates business adoption. Smart phones tailored to the business consumer have been tremendous business tools for over a decade, but the iPhone revolution surpassing their numbers can be attributed to a public audience of individuals, not business users. Because individuals are not limited by corporate policy and the need for internal studies, they are often the target for rapid growth models, not businesses. Cloud adoption will suffer because there is no key component that directly entices individuals as an audience in mass.

Another factor to the slowness of adoption will be the difficulty for which application providers can take full advantage of secure, elastic computing. In its infancy, Internet pages were easy and became a whole industry in and of itself tailored to those with high artistic skills. Cloud applications, to be elastic, will require new architectures for expandability and extensibility requiring highly technical skills to prevent CPU process run-away and high efficiency in all resource utilization.

Who Wins with Cloud Computing? As the diagram above depicts, I believe small & medium business are the big winners. They will very quickly adopt cloud IT services that will rival the best big business has to offer without the pushback big business will have to deal with (see Who Loses below). Low cost of entry and advanced handing of data will push them forward of legacy bound organizations. Think of it as how the cell phone infrastructure in India has completely bypassed landline operations in not only India, but even what we have in the USA!

Who Loses with Cloud Computing? Those who resisted change in the past (e.g., like back when COBOL programmers refused to learn Java) are the anti-pattern for Cloud adoption. Not only will those who resist change bring themselves into obscurity, they also risk taking their employers with them when they seek to justify their existence and preserve their legacy positions rather than adapt and redefine themselves for all the new needs that surround Cloud Computing.

The next group of losers will be developers unaware of writing efficient, robust, and fast code. Think about it. If you're paying twice as much for each CPU cycle, poorly written & optimized code will surface at the next billing cycle just as when you go over your minutes on your cell phone plan!

I genuinely look at this as a real cleansing opportunity to course correct what I might suggest as a lapse into sloppy coding practices over the time of ever faster hardware and ever increasing memory at low cost.

Are you willing to wait? It won't happen over-night! So, from an infrastructure cost analysis, it’s pretty easy to determine if Cloud Computing is for your organization. However, the tell-tale rate of adoption will really rely on how each individual organization accepts change, in general. Those great infrastructure cost benefits may too quickly be offset by the costs for necessary education to be able to build scalable applications for the cloud and the cost of promoting adoption to individuals not willing to change. Are you willing to invest in your development teams, learn new architectural patterns, and understand which business processes will benefit most from the cloud model? There are no shortcuts here!

So, what will it take? First and foremost, the cloud infrastructure must be built. No matter what anyone claims, it does not adequately exist today. This has never been more evident than with the recent passing of Michael Jackson. Google, Facebook, Twitter, etc. all experienced turmoil as utilization spiked with the announcement. What’s also interesting about this is the increasing number of events like the recent Iranian election that cause tremendous load on the internet in the form of spikes. Simply put, the internet is not yet elastic!

Once an infrastructure does exist, I believe organizations have their own hurdles to overcome:

Building elastic apps is going to be different - As stated earlier, development will now need to take into account performance, memory footprint, etc. as the cloud is a pay-as-you-go platform. This also includes creating applications that easily expand in a cloud environment. Design decisions around state management, coupling, synchronous/asynchronous connectivity, etc. will need to be considered. Bottlenecks must be identified and overcome. If I put my application in the cloud but host my data in my own infrastructure, I have a serious bottleneck to consider. All these will require serious thought in the architectures selected for cloud-based applications. Take these considerations and weigh them against the limitations of each vendor’s cloud offering, and you have a starting roadmap for your future development burdens.
Data storage models must change – The way we store data today is not cloud secure nor cloud efficient. We build gauntlets around our data in our data centers to prevent any breach of our vital information. This in and of itself becomes a bottleneck as stated above. In our current systems, the single point of failure is usually where the presentation and business layers connect to the data layer which is often at best active/passive in its redundancy. I see real opportunity for data being cloud-hosted in an elastic manner when it is first encrypted and then stored in the cloud. This protects the data no matter where it ultimately resides in the cloud. Today, a government inquiry to a hosting organization may mean all your data is freely given up by that hosting organization without your consent. With cloud data storage that is pre-encrypted, it doesn’t matter if this happens. Should the data find itself in the hands outside your organization, it will still remain secure. To do all this, however, storing and efficiently accessing data can’t be done in the relational models currently being used. New, elastic cloud storage models must be developed, understood, and implemented. Efficiency of accessibility will be a real hurdle.
Security will impose its own challenges/benefits – When looking at securing the cloud, I believe new opportunities exist to facilitate more secure access to protect data than what many organizations have internally. What is stated on a design vs. what is implemented, tested, and verified are often two different stories. If your cloud vendor’s data hosting capabilities meet regulatory requirements, you are way ahead of the game. The cloud also introduces risk. Even if my data is pre-encrypted, the key to my data is still subject to being compromised, or worse yet, lost so my data becomes completely worthless. There is going to need to be a whole new set of security practices and processes to govern the keys to the data just as they exist today around the data. Authentication into the cloud will be as easy or complicated as we make it. Most likely, we will make it overly complicated.
Automated porting of apps does not make them cloud ready – If you’ve ever migrated from Lotus Notes to Exchange and attempted to migrate your Lotus Apps to InfoPath apps using an automated tool, you may find that you were very impressed with the demos but highly disappointed with the results when put into practice. This is not to pick on migration tools, rather it’s to pick on the idea that there is value in migrating everything through automation. Apps in the legacy platform should be reassessed for the value they offer and then be migrated (using both tools and smarts) to the new platform in an intelligent manner that embellishes the new platform’s capabilities. I would look at the cloud platform as the platform for future developments initially as green field apps are a better investment during the educational ramp-up. Once you obtain a level of cloud-platform maturity, you can then best identify what legacy applications make the most sense to migrate.

Starting sooner is better than later: A wait-and-see approach is not going to buy you anything with Cloud Computing. Don’t get me wrong, I’m not suggesting migrating everything to the cloud now. Rather, I’m suggesting that you consider beginning the education process now. Create an account in each of the cloud vendor platforms and start learning how to make your applications elastic, play with scalable data storage models, and learn how to monitor your hosting services to validate they are operating in the guidance of their Service Level Agreements (SLA).

Those who become early adopters will also be those who shape what the wait-and-seers get. Think about it. If I sign up for BPOS or Google apps and don’t like it, I can be part of making it work the way I do more now than later.

Happy Cloud Computing!

More Thinking on 3D Thinking

Jim Wilt — Tue, 09 Jun 2009 22:10:00 GMT

After some great feedback email and conversation, I'm going to elaborate a little further from my previous post. First and foremost, my colleague, Max pointed out the tangent is equation is dy/dx not dx/dy (I am often suffering from dyslexic tendencies in both writing and speech). Next, Max reminds me setting it to zero is for finding the horizontal tangent. Please don't tell my Physics professors I messed up!

Next, everybody I know named, Andy, brings up questions on thinking beyond 2D and 3D. What of 1D, 4D, and beyond?

Let me start by qualifying that my response may be more conjecture than scientific fact. I trust those much smarter than me will correct my thinking with the solid facts :-)

Now, let me try to classify the dimensions:

1D	Acting without thinking. When someone cuts you off in traffic and you release an explicative, they can't usually hear you and your explicative doesn't correct their behavior, but it sure makes you feel good.
2D	Using logical inferences to draw a valid conclusion (a line from before). If the sun comes out today, then it will be warm. Note, not all valid arguments are sound.
3D	Using valid logic and in pursuit of true premises to draw a sound conclusion that may not be absolute, but possibly a framework that encompasses many (ideally, all) valid perspectives (a plane from before).
4D and beyond	Beyond logical inference to creatively discover truths that would otherwise be obscured by logic alone. Einstein with his many theories is an example here.

Being a 2.5D thinker at best, I do not believe multidimensional thinking is linear in any way nor do I propose a pragmatic path to get oneself there. I simply am basing my conjecture on observation of those whom I observe to practice the art. Consider the non-linear curve as shown:

Stacked side by side:

This might imply two things:

Multidimensional thinking has a “sweet-spot”
There exists a threshold – which I refer to as the “threshold of human comprehension”

The latter imposes a whole new debate. There are those who would imply anything and everything can be explained, however, due to what ever limitations exist in our complex minds, we cannot comprehend everything.

l might suggest that individually, our personal threshold, moves up and down as we learn, mature, grow, etc. However, I would also suggest that while our personal thresholds are relatively dynamic, there exists an absolute limit to what we as biological beings can comprehend. I firmly believe we are yet to even come close to seeing that threshold, but there are times when we see a glimpse of it and have to take things "on faith".

Why must we insist on solving 3D real world problems with 2D solutions?

Jim Wilt — Sun, 31 May 2009 21:24:55 GMT

A really big frustration I have is when my colleagues who genuinely think in terms of 3D solutions are readily discounted by stakeholders who only want to hear 2D propositions because 2D thinking is so much simpler. Let me illustrate:

With oil prices about to rise, the debate over consumer choices for transportation will once again surface at its usual passionate if not religious level. Individual transportation that is cost conscious and Eco-friendly really is a 3D problem. The debates stem from competing 2D solutions being pushed as solutions to this 3D problem.

Graphically, it looks like this:

To find the tangent line for a continuous curve at a given point, you take the first derivative and set it to zero. There is only one tangent line to the curve at that point in 2D. Period. Thinking in 2D is simple… The math is easy and it is simple to illustrate for even a non–artist like me.

My lawnmower needs a half liter of petrol to mow my lawn, therefore, I must purchase at least a half liter of petrol before I can mow my lawn.

2D problems with 2D solutions fit together perfectly and they are easy. That’s why we like them so much.

3D problems, however, are much more difficult:

Above is my feeble attempt at drawing a continuous surface curve in 3D which we will use the point to represent the real–world 3D debate over consumer choices for cost conscious and Eco-friendly transportation. Because most of us want to keep things as simple as possible, we jump on a given 2D bandwagon that satisfies at a more emotional level like the following 2D derivatives taken against the 3D surface at the given point:

H – is a tangent line that represents the passionate drive that Hybrid vehicles are our salvation
E – is a tangent line that represents those who believe there is only resolution in the use of purely electric vehicles as they require no petrol at all
D – is a tangent line which represents those who know Diesel vehicles are the answer because they can operate on pure vegetable derivatives of fuel

Notice that each of these is a valid 2D solution (each is a line that is tangent to the curve at the given point) and yet none are totally aligned with the others. In fact, mathematically, there are an infinite number of lines tangent to the given point just as there are an infinite number of 2D cost conscious and Eco-friendly transportation solutions to be passionate about (e.g., walking, cycling, buses, trains, etc.).

Wow! If there are an infinite number of 2D solutions to a 3D problem, is any one of the individual 2D solutions really any closer to solving the 3D problem? Quite possibly not! They may give us an illusion that we are solving the 3D problem and may even make us feel good for awhile, but soon reality sets in and we see that the true 3D problem remains. We all drive Hybrids and we still run out of fuel; we all drive electrics and we can only travel 40 miles round trip maximum; we all drive Diesels and we no longer have vegetables to eat.

So, what does a real 3D solution look like? The answer is as simple as the math: An infinite number of lines tangent to a given point on a continuous curve surface is the definition of a plane!

Again, apologies for my feeble attempt at illustrating this. The fact is that when facing a real-world 3D problem, we should be seeking a real-world 3D solution, always. This means we are looking for planes, not lines.

The solution to the cost conscious and Eco-friendly transportation problem is found in the rewarding through incentives for the use of any alternative that has a lower carbon footprint, in general, and not selecting any given alternative specifically. Further, education would promote that any of these are contributing to solving the problem from the same plane and nit-picking one over another is just plain silly.

The same can be found true in our IT architectural problems and solutions. We too quickly are looking for simple, cost-effective 2D products to solve our 3D problem spaces when it would be better to define a 3D solution framework from which multiple 2D product solutions can coexist.

Sometimes, the 3D solution is just too hard to find, even for the best 3D problem solvers. How do we deal with this?

Reality Check: 3D solutions are at best much harder than 2D solutions to solve. Even if you are capable of solving 3rd-degree partial differential equations, it will take you exponentially longer to do so than if you did it in 2 dimensions. It really is that much more work!

We can't always invest the time and money required to "do the complete math" so we often choose instead to rely on close approximations.

I can recommend that in these situations, you seek individuals who represent the best 2D solutions in the given problem space to team up and build a 3D approximation framework using the following guidelines:

The individual 2D views must be vetted as a valid 3D approximations
Pick individuals whose 2D perspectives are more orthogonal than parallel - as this will lead to an approximation framework that is more complete allowing for greater interpolations
All individuals must be on the same plane or your approximation will be invalid
Because the 3D approximation is an approximation, it will never be the complete solution, so don't treat it like it is; stakeholders must understand this

I think being on the same plane is paramount. Many will state they are on the same plane, but their own agendas will place them on completely different planes. Learn to recognize this!

As an example, look at our multi-party government's view of national health care reform. All parties will state that because they want reasonably cost-effective and all-encompassing health care they are all on the same plane. However, because of hidden agendas where one party wishes to outshine the other, for example, there is no common plane, and efforts to produce a 3D solution approximation continue to just be a battle of 2D solutions from different planes.

I've had opportunity to work with some of the best 3D problem solvers of our time and must say that when following the 2D approximation model where are all coming from the same plane, I have seen 3D approximation frameworks produced that really make a difference.

What is the source of this common plane? Usually it stems from a common business goal, strategy, or shared passion for the betterment of others.

Case in point is all that is being done to promote the education and practice of IT architecture. From IASA to The Open Group to the MCA, a common plane exists around making IT solutions better through the best holistic practices in IT architecture. While there are those that look at this promotion of more 3D thinking and problem solving skills to be too much to ask of those interested in expanding their horizons around IT architecture, these organizations have never lowered the bar nor accepted 2D practices to be acceptable substitutes in this 3D problem space. While many 2D thinking critics take their stabs at their inflexibility, I can only appreciate their true understanding of the 3D problem space they are defending.

That said, I come back to a variation my title. Why does the world insist we must solve our real world 3D problems with 2D solutions? Why don't more of us stand up and identify how silly it is to belabor around an infinite selection of 2D solutions? How long do we tolerate 2D solution seekers for 3D problems who can never be satisfied? Lastly, how do we promote 3D thinking in a world that labels those who promote this as ivory tower?

A Tribute to Bob Fry

Jim Wilt — Mon, 11 May 2009 12:10:00 GMT

I spoke at TechEd2009 several times this week and after an intense day of architectural sessions, some industry and Microsoft speakers for the architect track had dinner together. This was really cool because I was able to listen to some of the best and most influential minds of our time.

Through our conversations I mentioned one of the most influential professors I ever had probably lost his position based on my blistering review after completing his class only in later years to see the brilliance of his genius. My colleagues encouraged me to blog about this, so, I resurrected an old paper where I pay him tribute.

My tribute to Bob Fry is embedded below in a paper I wrote in November 2004 which, while somewhat dated, can offer itself as a catalyst for further investigation. Sorry about the goofy formatting. It originated in Word and translating it to HTML that works across all browsers is... well... actually a topic in the paper! (in other words, if it doesn't look proper in your browser, try another, it most certainly will look different)

Sobering* Words for Computer Science Academia

a white-paper by Jim Wilt, November 2004

*marked by temperance, moderation, or seriousness; showing no excessive or extreme qualities of fancy, emotion, or prejudice – Merriam-Webster

Sobering Words for Computer Science Academia – Overview

This paper is being addressed to higher education institutions and those organizations that set curriculums for Computer Science degrees. It is intended to offer suggestions towards the better education of students bound for Computer Science development careers -- from an industry point of view. It is a culmination of experience and interactions with industry and candidates entering the Computer Science Workforce.

Innovations in current curriculums are identified and some alternative approaches in the education model are suggested. There are many influences that direct the manner in which education is approached that are too numerous to enumerate here. The approach in which I propose to respond to these influences comes from a sign found years ago in my daughter’s 7^th grade science class:

Don't make Excuses,

Make Improvements.

Opportunities to work and collaborate with many individuals in an intensive software development setting have identified that many professionals are coming out of our educational institutions with Computer Science degrees having an inaccurate perception of industry while lacking necessary attributes to achieve key skills.

Industry needs (from this author's point of view) are being used to provide feedback to the academic institution for the betterment of the Computer Science student. They can be summed up with the following observations:

Listening, Verbal Communication/Presentation, and Problem Solving Skills will take you much further than training of any specific operating system, language, or piece of hardware.
In the ever changing world of Computer Science, the student must learn to learn new technologies, adapt to ever changing languages & operating systems, and mostly, learn to use/adapt proven modern techniques of development methodology in an industry that sometimes frowns on them.
The Experience Paradox can be alleviated by an Off-Shore On-Campus working model.
Finally, the business model used by most higher education institutions, as well as many public education systems, needs an overhaul. The model where the student and their parents are the customer pushing universities to provide a student-centric environment is in need of change.

A New Business Model

To maintain a much needed cash flow, many higher education institutions have adopted a business model where the student is the customer, requiring faculty to cater to student wants and needs. This is ever evident when you walk on the campus of a major university on a Friday morning or afternoon and find less than 25% of the classrooms in use.

Administration must deal with funding cutbacks, passing tighter budgets to their various colleges and departments. College/department budgets key performance indicators are being directly tied to:

The number of students enrolled in each college/department.
The evaluations filled out by students on their professors.
The number of students graduating from their undergraduate programs.

The following are flaws in this model:

Not all departments can compete with the numbers of others due to the nature of education involved. Physics major numbers, for instance, their will never match that of those in the Business programs. Just the same, it may be more expensive to teach a Physics major due to the expense of equipment in labs, etc. compared to other departments.

Sometimes, it hurts to learn. Many students will produce a negative evaluation of a professor with whom they were frustrated or feel did not do justice to the material --- until after a few years in industry they find that that professor taught them more than they ever could realize. Many Professors feel stifled due to the fact that their livelihood is directly tied to student evaluations.

Note:

One innovative department in the College of Science and Technology at a major university has better addressed the effectiveness of evaluations by taking them 3 times: (1) at the time of the class, (2) one year after graduation, and (3) three years after graduation. This provides a more true measure of the class’ effectiveness.

The counting of students graduating from a given program is a most unfortunate metric. It encourages pushing students through a program that otherwise should be filtered out. This adversely affects industry because it is inundated with a pool of both genuinely capable candidates and those who will ultimately render themselves as incapable.

When quality is measured by quantity, quality is the first thing to suffer.

A replacement model being suggested is as follows:

A New Higher Education Business Model:

Science & Industry are the customer.
The Student is the product.
The Faculty is the developer.
The Administration is the project management.

Under this business model, Industry has some sobering words for academia:

Industry is disappointed with the quality of your product (students).
Industry would like to see management (administration) take more responsibility about this. Industry would like better access to management. Management rarely, if ever, calls the customer (industry) to find out how their product suffices Industry needs.
The only way your development staff (faculty) can hope to produce a better product is to give them more freedom in their domain to develop with proven methods and further to encourage them to learn new development methods and technologies involved with producing a better product.
Finally, management appears to be focused on short-term gains. Industry would rather the focus be a long-term investment towards a better product-line by increased interactions with Industry.

In a sense, the quality of students today can be related to that of late 70's automobiles, an era where the product suffered from rust, pealing paint, and mechanical problems. Management and development blamed each other for their failing product. The threat of a higher quality, lower cost overseas products loomed over the viability of their product.

Higher Education may find the key to producing a better product is putting development staffs armed with more applicable curriculums back in charge of the product supported by enabling management that better directs from more frequent communication with its customer.

Putting Science back into Computer Science

Most technological/scientific fields find their pure roots back at the university where absolute objectivity, truth in reporting, and meticulous measures of detail oriented research are taught and religiously followed.

It appears to be a polar opposite when the field of Computer Science is discussed. Opinions (sometimes religious) about platforms, languages, and sloppy practices abound. Attention to detail and objectivity sometimes appear non-existent. How can this happen?

Perhaps it is due to the field of Computer Science being such a young field. Maybe staffs many times do not come from backgrounds as disciplined as the mainstream sciences. Could it be that classroom projects focus on the deadline and not the technology, methodology, or architecture?

One must question that as less-than-objective/scientific practices work their way into industry, is it any wonder that projects suffer from cost, time line, and quality issues? The only group in industry actively pursuing the use of Business Intelligence and detailed analysis of metrics tend to be those from financial disciplines (Accounting departments are using more technology to measure their effectiveness than IT departments). Without accurate metrics for the science of this industry, it will continue to be led by cowboy development methodology and shoot-from-the-hip management practices.

We need to bring Science back into the field of Computer Science.

Problem Solving Skills - Where Are They Taught?

Most Computer Science programs are related to the specifics of the computer, its architecture, and some exposure to software methodologies. Where is the development of Problem Solving Skills?

At the undergraduate level, there are few if any courses tailored to the art of solving problems. Where and how can a student expect to become a developer without proper training in the art of solving problems?

One answer typically offered by educational institutions is that it is found in required courses from other departments (typically Mathematics). While these classes are beneficial, the required supporting classes often do not venture deep enough. I will illustrate this by example:

Advanced classes equate to better problem solving futures:

Calculus III (not typically required by a CS major curriculum) is more applicable to a developer even if he never uses it directly in his career.

How is this?

Simply, Calculus I and II are pretty much limited to two dimensions with most problems requiring several minutes to complete.

Calculus III takes the attributes of Calculus I and II to three dimensions adding partial differential equations. These problems may take many minutes to hours to complete.

What's the benefit? The benefits are threefold.

The student is forced to push their thinking process beyond the obviously tangible into more abstract thinking.
They are taught to persevere when attacking a problem and find that all problems do not have a simple answer. (These are the problems with which industry deals )
They learn to sometimes work a problem backwards, sideways, and from other angles to identify the correct forward algorithm.

Calculus I and II challenges prepare a programmer to write an application involving a 2 dimensional array.

Calculus III challenges prepare a developer to design a system around a 64-dimension OLAP Cube.

This problem permeates well beyond the first few years after graduation. I have been exposed to some brilliant minds in this industry – many with 5 plus years experience. My interactions with them have been (carrying the example metaphorically) to mentor them into 3 dimensional problem spaces from their historically 2 dimensional problem spaces. When discussing our solution scenarios (which are significantly more complicated in 3 dimensions than in 2) they follow along and even are capable of grasping the base concepts throughout the engagement.

However, after the dust settles and they are left to continue on their own, they are convinced that there is a far easier solution for the 3 dimensional problem spaces if they were to stick to their 2 dimensional techniques often abandoning the more appropriate 3 dimensional solutions.

Calculating the first derivative vector of a 2D quadratic equation in two dimensions yields a tangent line at a given point on a continuous curve.

Calculating the first derivate vector of a 3D quadratic equation using only two dimensions will also yield a tangent line at a given point on a continuous surface, however, there are an infinite number of other tangent lines in the same plane that are being ignored.

A developer not challenged or pushed into a more difficult problem space will fall-back and resort to their comfort zone – unknowingly placing their employer into higher risk and cost by creating limited scope solutions that may ignore an infinite number of more appropriate solutions in the real problem space.

The art of learning Problem Solving Skills and abstract thinking is undoubtedly a difficult one, but the individuals who delve into the advanced courses are certainly more prepared to take on the challenges that industry and the sciences have to offer.

One solution some universities currently offer in this area is that of the Interdepartmental and Interdisciplinary Majors and Minors. This is not a trivial task. It means significantly more work from both departments as each department is involved in the assignments of the student from different vantage points. These programs are most certainly a major step in the right direction.

I would urge any student that is serious about becoming a leading industry developer to make their Computer Science major a Computer Science-Mathematics, Computer-Integrated Manufacturing, or any of the other interdisciplinary majors.

Double-Up with Sister Departments

If an interdisciplinary program is not offered, or there is not one to a student’s liking, Industry finds that graduates with co-majors/minors in the main-stream sciences will often offer deeper problem solving skills. Additionally, these candidates often exhibit higher levels of objectivity, attention to detail, and effectiveness in their work.

I would ask Computer Science department student advisors to encourage their students to expand their educational focus to consider a second major or minor in one of the main-stream sciences including (but not limited to):

· Biology

· Chemistry

· Geography

· Geology

· Physics

· Engineering

· Mathematics

Problem solving skills are already embedded into the curriculums of these sciences – and problem solving skills most certainly are transferable outside their domain.

A Mathematics minor should be a requirement as it facilitates the beginnings for true abstract thinking and problem solving.

The Ability to Learn a Technology is better than Learning a Specific Technology

Computer Science and software development often lend themselves to the adoption of marriages. Marriages are often to a given operating system, language, or developmental methodology. While it is good for a developer to be excited about a given technology, it can often be harmful to a company's future growth if their development staff cannot adapt to new and emerging technologies. After all, how many of us can safely find an interesting career programming in PL/I today?

In these marriages, Hardware Platforms, Operating Systems, and Languages seem to be the technologies that have the largest romances. Over many years, a developer can become very comfortable with a given technology such that the thought of changing it is like that of getting a divorce from something that has been your tried and true friend over time. Sure there were ups and downs during that relationship, but overall, a given technology may become a true part of a developer's existence. Everybody knows Stan, he's that LISP expert. You can ask him any question about LISP or CICS and he's got the answer. But wait, we quit using LISP 4 years ago!

The point is that computer technologies change, many times from project to project. New technologies are introduced each year. Industry is fickle. Some industries are early adopters of new technologies and others are not (like the one who employed Stan above).

In other sciences, industry looks to the educational institutions to objectively evaluate new ideas and technologies, many times from graduate study programs. For some reason, Computer Science claims to be more of a victim of the ever changing technologies – not being able to keep up due to costs, equipment changes, education, etc. I should think that Computer Science departments would embrace these continuing changes as they provide an ever fresh pool of graduate independent study topics. Unfortunately, some of the largest push-back to new technologies comes from the educational sector.

Learning a specific technology is a beginning. The act of learning a technology is far more important than any specific technology learned. A software developer who adapts to the real-world environment of technological flux is far ahead of those who stagnate in a specific technology.

Current curriculums often are set for a specific technology and rarely support changes of technology mid-stream. Perhaps it’s time to modify class definitions from specific technologies and start including multiple competing technologies in the delivery.

Some might argue that this causes unnecessary confusion for the student. I would argue that this takes the student closer to reality, and further builds up a resistance and skill before it happens in industry.

Science is a Creative Art

I am a firm believer that one must learn the basics in any profession, pursuit, or venture. Having a solid backbone understanding of the qualities behind Computer Science you may better recognize when it’s applicable to apply a proven pattern for a solution and when its time to invent a new solution.

On occasion, a more difficult a problem will sometimes fail to resolve as a more logical approach yields no progress.

No matter how hard you try, a physical connection will have a bandwidth maximum. If your problem solution requires you to significantly exceed that maximum, by 10 fold for instance, then a logical approach may lead you to the use of some form of data compression. Now, if the information you need to transport is already compressed, logic begins to leave you feeling a bit stale.

A more creative approach is an elixir to the software designer. The greatest discoveries have come in the form of a creative approach. Taking a problem, turning it upside down and inside out often releases a solution that deviates far from the norm but delivers knowledge to a new level.

One might attempt to argue that creativity cannot be taught. Either you have an affinity for the arts or you don’t. I am one to argue that it is an essential required quality of the Computer Science profession.

Close examination of a musical score identifies that it is truly the first ever programming language.

· It supports looping with conditional branching.

· If supports goto and subroutines (refrains).

· It is exact and repeatable. Programs (Songs) sound the same when replayed.

It is only natural to draw the conclusion that the basics in the art of music parallel the basics of Computer Science. Imagine what happens if we attempt to apply harmonics and rhythm.

Just as I’d hope a Computer Science curriculum would require a Speech class (taught by the Theater dept.) reinforced by regular oral and written presentation, I would argue equal merit to the requirement for a form of art, humanities, or music likewise reinforced by creative solution exercises.

When in High School Physics, my class was given the exercise of finding the acceleration of the elevator in the courthouse. Did you get that? The elevator (small town). Pairs of students would be seen taking a bathroom scale into the elevator to measure their difference in weight as the elevator rises and falls to calculate the elevator’s acceleration. My lab partner and I decided on a different approach. We used a Tennis Ball and a Stopwatch. Even having a measurement that was no where near the actual acceleration, we acquired 15 points on a 10 point assignment because our approach, although yielding a far from perfect answer, had never been tried and theoretically offered as accurate a measurement under tighter tolerances.

Creative solutions are identified by example, nurtured by discounting the obvious, and established by attempting to solve the impossible.

Look for Partnerships to Augment New Technology Education

Not just students, but Industry also needs classes developed that relate to the new, emerging technologies - many of which are not yet fully developed. This poses many problems for higher education's faculty:

· How do they teach a technology that is in its infancy?

· Where do they obtain the education required to become knowledgeable enough to teach a given technology?

· Where do the materials come from which they can teach these new technologies?

Many professional education organizations for industry are developing courses and materials for new technologies as they are being developed through partnerships with the publishers/vendors of the technology. Seek partnerships with these organizations and include their training courses as part of your curriculum offerings. Further, these courses will bring in industry revenue as professionals in industry also seek training. Many times, travel is required. Having the technological training available at a local university makes it much more attractive.

Additionally, I recommend that teaching staffs regularly attend training opportunities whenever possible. It offers not only benefits from learning a new technology, but also exposure to industry professionals that attend these classes. Positive exposure to industry often comes in the form of networking at these classes.

Methodologies, “Process vs. Results”, Reviews, & Testing

Methodologies

Jumping in and coding before business processes are defined, documented, or understood seems to be a common theme in education and industry. Technology providers too often are no help here. In their one-day seminars introducing a new technology, they show you how to put a complete solution together through drag-n-drop in 5 minutes with no code that used to take “hours and thousands of lines to develop.” It makes for a great demo, but obfuscates building a solution around demonstrated user needs.

This is where a Computer Scientist needs to realize that immediate attention to detail backed by following a proven methodology will be the best time saving investment for the duration of a project.

It is so important that architects and developers understand business needs, rules, and value before implementing a given technology. Methodologies allow architects and developers to identify, document, and communicate this with their end users.

Development of systems using a proven methodology, or process as it is often called, results in projects coming in on time and with fewer problems that generally plague the sit-down-and-code method. There are different methodologies in this domain, and none if them is appropriate for every situation. Project duration, customer/internal department end goals, and cost all play in the determination of which is the best fit. Clearly defining the methodology to be used for a given project, and communicating that with the customer/end user is paramount in a successful engagement.

Just as one language cannot accommodate all software solutions, there is no one single methodology that will work for all development efforts. From waterfall to solution framework based to agile, there is an appropriate methodology to be chosen based on project size, budget, and timeline. It is just as important for students to understand how and when to use each as well as to be able to adapt to the many variants that arise during each organization’s practical use.

As we continually evolve in industry to more use of software engineering methods and tools, shouldn't students be practicing them from the get-go? It is imperative that students make regular use of, and not simply have exposure to, software engineering tools and techniques. By introducing these concepts prior to the actual language classes (they currently show up at the 400 level) followed by repetitive use in subsequent courses, curriculums can better encourage acceptance and mastering of methodologies.

As marriages are often to one’s first exposure in Computer Science (Fortran 66 for me), wouldn’t it be better to court a methodology first? This may lessen the tendency to become tied to a given language, OS, or platform.

The Process vs. The Result

Too often students are more interested in the end result of their assignments than the process used to get there. Why is this? Could it be that the educational evaluation mechanisms place too much importance on the final product, rather than the work done getting there?

In main-stream sciences, the process to the end solution is scrutinized. One of my Physics tests was to solve the Schrödinger equation for a person walking through a door at a given velocity (it’s normally used for sub-atomic particles). The reason the instructor chose to use macroscopic elements (people – yielding numbers which were unbelievably obscure) was to see that we understood how to solve the equation, not simply get the right answer for an antineutrino.

The evaluation of a student’s work should not be focused solely on the solution, rather, it must be more so focused on the process used to arrive at that solution.

Starting a Computer Science Major with courses such as a Principles of Computer Programming which addresses algorithm development and problem solving methods is a bold and well supported move in the right direction. A repeating theme is that one class is not enough. The concepts and principles must be repeatedly used and reinforced in all courses that follow.

Reviews

An important aspect of development in industry that appears to be missing in curriculums is the process of proactive Code Reviews. Students need to realize their code will be reviewed by their piers. It is better if they experience this from their course work.

A code review in industry is not meant to be a judgment of a person's abilities, but rather to be used as an effective tool in spear-heading problematic areas. They also are effective teaching tools as the developer being reviewed can share what new innovations they learned in the process of their development with their peers.

Testing

Test-Driven Development (as a methodology), Test Plans, and Software Testing are additional tools that appear to be neglected at educational institutions. There is an established science in industry in which products are tested. This involves a methodology for developing test plans and procedures which check for completeness, consistency, usability, repeatability, and other aspects of a product. Automated testing programs and scripts are used for regression validation, stress tests, and identification of load capabilities for a given solution.

Testing has the highest potential of truly advancing the Science of Computer Science, however, it is probably the least utilized in both education and industry. Very little is known and taught about effective testing practices outside of isolated industry islands.

Methodologies and Testing also are in a state of continual change as they evolve. Educational institutions may feel that they can’t keep pace, but my question is why aren’t they the leaders here? Shouldn’t our educational institutions be offering new theories of development and testing to industry?

One leading research university has developed a methodology on designing for usability and is using industry leaders to evaluate the effectiveness of their program through evaluation of work from students who have taken the class. This should be a more common norm and not the exception. This is where a university is taking a proactive look at the industry and trying to improve it through better science, but testing its effectiveness with industry evaluators. Awesome.

It’s all about Communication

There are many aspects of teaching Computer Science related to the ever changing developmental methods, operating systems, and computer languages.

Many students demonstrate technical skills from a sterile lab environment where they might create a normalized relational database for an imaginary company. While these serve to assist the student in working with and learning the technology involved, they ignore that a large part of the problem in developing a solution is the Needs Analysis and Communication with the end user.

There is a true need for the student to perfect more than just the technology involved in implementing solutions - that which appears to be the main topic of most 100-300 level courses. There needs to be sufficient training in analysis of user needs, communication techniques to identify these needs, and documentation skills to recap these determined needs back to the customer/end user.

It's a rare situation that a customer or internal end user department provides a sheet of paper with instructions to develop a solution for them using a star or snowflake schema to track all grocery stores and their products with no more than 10 balanced and normalized relational tables. Rather, they say they've got a lot of stores - more stores than anybody else (in their opinion) and they carry more products than you can imagine - finally they want to know which store has how many of each product at any given moment in time.

This is where Listening Skills come into play. End user needs presented over a white-board session given verbally, and inconsistently, relate more to the Real-World. In 4 years of Computer Science classes (from 1978-1982), only one professor, Robert Fry, gave his assignments orally. He would change his mind mid-way through, and often be inconsistent. Not until a few of years after graduation did I realize the true wisdom of his ways.

What Bob Fry would frequently tell his students was that they had to ask him questions to get details that he might leave out. A skill required by a developer is that of asking the right questions, in different ways so that you obtain all the facts. While hashing through a problem with the end user they will often change their mind as they are forced to look at their problem from multiple perspectives.

Sound Verbal Communication and Presentation Skills are the foundations required to effectively present ideas in a group setting. In the presentation of architectures and implementation plans, a student must communicate their solution is based on a sound understanding of the business needs. They must also be prepared to interactively take challenges of their logic - and learn to adapt to those challenges.

Do Computer Science departments think a general education speech class will fulfill mastering of communication skills? These fundamentals must be reinforced by continually exposing students to oral examination of their ideas with interactive discussions examining their solution proposals. Only through repetition will students overcome their natural fear of speaking and refine this much needed skill.

Communication in all its forms is an important part of the Science in Computer Science.

Instill Discipline

What I would venture to say is non-existent in curriculums is that of Development Discipline. Why instill a discipline? It is one of the fundamentals of our science. The definitions of coding standards, methodologies, and practices alone only have value when they are used. Consistent use of sound practices results in higher quality output, in less time.

5 Coding Myths:

I can save time if I take a short-cut now.
I’ll go back and clean it up it later.
If I do it the right way right now, I’ll fall behind schedule.
This code will never be used again.
I don’t need to learn another language/framework/OS. The one I know will be around for ever.

Development Discipline is not about coding standards, it’s about applying coding standards. It’s not about design methodology, it’s about consistently implementing a good design. It’s not about setting up test plans, it’s about using test processes from the start.

Development Disciplines can be summarized into three major categories.

· Syntactical Discipline centers on making and consistently following coding practices that are sensible and promote the effective development and readability of code.

· Architectural Discipline centers on the design of an evolving framework to eliminate code duplication by utilizing language technologies (e.g., Objected Oriented Inheritance) and patterns. It’s a practice of understanding the design and identifying queues during development for an architectural iteration that will extend a framework as required.

· Testing Discipline centers on utilizing your development platform’s monitoring, debugging, and test capabilities to thoroughly validate code by scientific evaluation of execution flow, exception handling, memory utilization, and performance metrics.

Development Discipline could become a new course offering where these core values are established and practiced repetitively through a series of assignments.

Industry Requirements: Professional, Responsible, Accountable

While message boards and journal article quotes touting subjective opinions related to various technologies (platform, OS, and language choices) are sometimes entertaining, architects and developers are expected to exhibit behavior that is:

Professional.
Responsible.
Accountable.

Period.

Credibility in industry can only spiral downward when one starts letting their marriages surface as they start making subjective remarks about products, processes, and technologies.

Likewise, the educational institution must uphold to higher standards of objectivity. Naturally, teaching staffs will have their own preferences and biases toward one technology over another. The classroom, however, is not the place for these preferences to surface. One might argue that even during office hours, a higher level of objective professionalism could be obtained.

Back to a Physics analogy: Some of the best professors I’ve had, when pushed to take a stand on a given philosophical or technological debate, would throw the question back to me with, “what do you think; how would you test your theory?” Once again, science is used to establish credibility and personal opinion/bias carries no weight.

One of the areas where professionalism, responsibility, and accountability are challenged is that of Operating Systems and the Open Source movement. If one can move beyond their internal biases, there is ample opportunity for universities to take full advantage of the political, even religious, environment and start applying science to the debates.

I would hope to find more educational intuitions creating objective research publications on performance comparisons, total cost of ownership evaluations, extensibility and compatibility of solutions, etc. for the proliferation of platforms, operating systems, and products at large. Their objectivity will come under certain fire when they publish their results, but that’s exactly how you ensure objective scientific process are used. Further, the practice of defending their findings will create a most valuable skill when entering industry.

The real challenge here, however, is beyond that of objectivity. It moves into the realm of industrial appropriateness. You see, most academic institutions are under license agreements that differ drastically from Industry. While Red-Hat may be free to a student, there are real license and support costs to industry for the same product. This is where Responsible and Accountable come into play. Research must take into account burdens from an industry perspective as well.

Real-World Open-Source vs. Platform Portability

While in a strategy meeting with a CIO and executive management staff of a Fortune 500 firm, it was identified that all future endeavors were to be of open-source. I asked the question as to what this meant, and there was a strong assurance that open source was going to ensure a higher level of security/comfort. I pursued this further by asking if they ever changed the source of their open-source providers. The response was an overwhelming, “no” as that would change their license agreements and force them to have to support OS source in addition to their application source, requiring additional maintenance for OS upgrades. The real issue, from further discussion, was that they were under the perception that open-source was analogous to platform portability.

An invalid assumption from academia is that a class taught on a specific open-source OS and platform will be seamlessly portable to other open-source operating systems and platforms.

The reality here is that any change of platform and even from one version of an OS to the next is subject to incompatibilities. For many years I made a career making POSIX compliant operating systems compatible with each other at the POSIX API level. Similarly, SQL on Oracle is not necessarily the same as SQL on MySQL, SQL Server, or Informix. Each product vendor (open source or shared source) offers value-add for their product which might limit portability of solutions – unless – you code specifically for all platforms.

Coding for portability on multiple platforms is a most frustrating experience, and it is missing in academia. This is vital in that every developer is eventually forced into a new platform, OS, or language over time. The push-back I receive from universities is that they do not have the financial resources required to create a course around platform portability. I struggle with this argument because of the discounts that universities have available to them from technology providers. Isn’t open-source supposed to make products more available?

The damage to industry is that developers have a rude awakening when they are placed in a situation where portability is required. It can take 2 to 3 times the original development time allotment to make a solution work on multiple platforms. Porting solutions to multiple platforms soon becomes the sole focus and the actual purpose of the given solution is often lost in the busywork of making it portable.

After burying oneself in the portability battle, eyes are often opened to the idea that maybe some solutions are best left to a specific platform as they can be better optimized with significantly reduced development lifecycle times. This allows for energies to be focused on making the solution best address the business needs.

Bottom line – promote portability for what it is. Difficult, time consuming, frustrating, and done only when necessary.

Open-Source vs. Open-Framework/Architecture

The Open-Source movement has become more of a religious battle of ideals and has left the architect and developer behind in terms of true value.

Some technology providers have better interpreted the pain developers experience that open-source, in part, has sought to alleviate. They are now providing open-framework architectures in the form of application blocks of code. These are complete source code solution frameworks that developers can immediately use and extend in their solutions. Application blocks provide extensible interfaces to various critical functions of an application such as caching, configuration management, exception management, data access, and even client user interfaces.

A built-in flaw of a developer is the need to create everything from scratch for each solution. User authentication, error logging, data accessibility, etc. are all too often duplicated from project to project. The idea of reusable frameworks makes the most logical sense and is the goal of most development environments, but will go against the native grain of the developer.

It is especially difficult for academia to address this as very little code can be easily moved from the completion of one class into another. So, how can an educational institution promote the best practice of reusable frameworks?

A solution might be to set up a class around the implementation of solutions using several application blocks or frameworks developed at the university. Developing from a framework and not from scratch may be among the most valuable time saving learned-skills available to industry.

Teaching Real-World Programming

Discussed so far have been various aspects in support of Real-World development:

Software engineering techniques and developmental methodologies.
Listening, verbal communication/presentation, and problem solving skills.
Supporting multiple platforms only when necessary.
Utilizing frameworks.

A paradox that plagues college students looking to find gainful employment in industry is that of needing practical experience. But how do you get practical experience if you’ve been taking 20 credit hours of classes for the past 4 years?

What if there was a course (geared toward undergraduate seniors) titled, "Real World Programming" that brought all these aspects together?

The book, "The Deadline" by Tom Demarco could serve as the base for one of the most effective classes higher education could offer its senior level undergraduate students.

In this fiction, the author takes you through the development of large scale systems with different types of development staffs.

It accurately depicts many of the Project Management issues that are raised during the development of a project. There are personality issues with developers, unrealistic requirements from management, and unthinkable deadlines.

The following summary outline for a real-world programming class is offered:

Real World Programming

based on the book "The Deadline" by Tom Demarco.

The first 2 weeks are spent reading and discussing the book, "The Deadline" by Tom Demarco. Project life-cycle best-practices should be covered during this time.
Students are placed into 4-5 person development teams - pitted to bid on a project against each other.
Students development teems will individually meet with the Customer (faculty) who requests an application to be developed via a white-board discussion.
- Questions and answers from the development teams will be required to obtain all the requirement facts needed to develop the application.
- Student development teams may schedule follow-up meetings with the Customer to refine the details (more points if they do this).
Students are given one week to put together a proposal to sell their solution to the Customer.
- Effective communication with the Customer is encouraged and should be rewarded with more points in their favor.
- Wireframes & prototypes are beneficial during this time.
- User analysis and business processes documented/modeled with CASE tools are also effective.
Each team is given a 2 hour period to privately present and defend their design. In this session, they must:
- Identify platforms being supported and their choice of development tools.
- Present applicable algorithms and technologies to be used.
- Demonstrate any applicable prototypes.
- Negotiate a timescale in which the student development team can develop their solution.
- Define any Penalty Clauses for failure to deliver on time (e.g., loss of points).
- Negotiate a means of payment from the Customer.
  - Student Advantage:
    - Payment by each member by the hour.
    - Paid incrementally with incremental deliverables.
  - Customer Advantage:
    - Fixed bid.
    - Paid at delivery of final product.
- Final grade will be affected by cost overruns.
- The Customer (faculty) will schedule weekly review meetings to obtain progress reports from each team.
After a set time schedule and price is established, the student development team sets out to develop the product (over the next 6 weeks).
- Use of software engineering tools and development methodologies will be a key part of this phase.
- Use of Peer Code Reviews will add points to their project.
- Use of a Test Plan, iterative development, and unit testing of completed modules will add points to their project
During the development, the process is interrupted by the Customer with a CRISIS (approximately 2 weeks into the development phase).
- The Customer will add scope to the project. Additional features are required or the whole deal is off.
- The team will be forced to accept the scope-creep.
  - If they negotiate added hours and/or additional payment for the scope-creep, then points are added.
  - If they fail to get compensation for their additional needed efforts, they lose points.
Further down the development cycle, the Customer will call another crisis meeting (approximately 1 week after the previous crisis).
- The Customer will make a minor change a portion of the design based on newly discovered needs they have.
- Additionally the Customer will pull in the deadline by one week (they have a trade show and must have the product for live demos).
- The student development team must, again, be forced to accept the added requirements.
  - They now must gain or lose points based on how they handle this added scope creep.
  - Give them points if they attempt to phase in their delivery such that they deliver minimal requirements by the new deadline, but extend their original deadline for a later release of the final product.
  - Subtract points if they just buy into your demands (unless of course, they deliver with the changes on time - which may result from excessive padding).
Upon the deadline, see how each student development team pans out.
- Evaluate effectiveness of their choice and use of development methodology.
- Evaluate effectiveness of their listening communication skills.
- Evaluate effectiveness and quality of their presentation and verbal communication skills.
- Evaluate their use of Peer Code Reviews, iterative development, and utilization of unit and final test plans.
- Evaluate how their product turns out.
  - Does it meet Customer expectations?
  - Is it robust?
- Did they come in under, at, or over their budget?
- Did they meet (or exceed) user negotiated requirements?

While this class would be most painful to take, it will offer a sense of realism related to the actual development process.

Fully realize, the benefit of this class is not the final product produced by the team. Rather, it benefits from the following:

Students are exposed to the many real non-programming administrative tasks related to the development of a product.
The brick walls they are forced to run into and the related pressures associated are real.
They will learn that not all team members will hold up their end of the work, and others may have to step in and do additional work.
It allows the faculty to experiment with different situations over the course of teaching the class many times.

The Experience Paradox

Every senior undergraduate has experienced the Experience Paradox. The most disappointing words on a rejection letter from an employer are, “…unfortunately, the position you interviewed for requires a candidate with 2-3 years experience.” What normally follows are shouts from students of, “How can I get experience if I can’t get a job to gain experience because I have no prior experience?!!”.

One of the most common pursuits of students to alleviate this problem is that of summer internships. Many of these work themselves into permanent positions. I would surmise that this is more of an exception based on the number of students applying for internship positions vs. those that actually get them.

There has been another deterrent to students in the form of the threat of Off-Shore programming resources cutting into their potential positions as entry level developers in industry.

Off-Shore On-Campus Working Model

Students are hungry for experience. Industry is hungry for commodity priced programming. Universities are looking to gain better industry exposure and recognition.

For me, this adds to a tremendous opportunity for what I term Off-Shore On-Campus programs that supply commodity level programming skills to industry while giving students that highly craved experience. An outline of this concept follows:

o The program must be offered to industry at the college or university level. That is to say, industry contracts Off-Shore On-Campus programs with the institution, not individual students. This means the university carries risk for delivery and quality.

o Academia might find it best to partner/sub-contract with consulting originations that can act as agents for their services to industry customers.

o The Off-Shore On-Campus program must be offered at the graduate level (3 credit-hour 500 or higher level course) as a paid class by the student.

o Senior level undergraduate students would be welcome to participate under recommendation from an advisor – remember, the university is carrying any risk in this type of engagement with industry.

o All fees acquired by the program and collected by student enrollment would serve to support costs associated with the Off-Shore On-Campus program.

o By students paying for this experience, you’ll have more assurance that the motivation for getting into the program is to gain experience, prove themselves, and make it not about the money.

o Students should expect to expend 16 or more hours a week for this class.

o Commodity work most likely will not be glamorous, but it is the experience and exposure that you must focus on. Set expectations with students appropriately identifying this is the first stepping stone onto the more advanced work that waits beyond graduation.

o Anonymity is required – students engaged should be working on “projects” with actual customer interfaces being managed and closely monitored by faculty. Part of gaining experience is the ability to protect industry intellectual property (IP) and protect corporate confidential information. Academia must respect non-disclosure agreements (NDA) made with their engaging customers.

o Whether contracted or not, students must be immediately engaged into a project – Repeat past engagements when not actively engaged with a customer – repeating appropriate twists and pressures. The students need not know whether theirs is an active or repeated engagement.

o Students are graded on their performance of delivery, quality, and adherence to best practices.

o Accurate records of engagements can serve multiple purposes from more accurately repeating engagements to reporting metrics of cost and scope-creep back to organizations.

o Be responsible and incorporate all development best practices:

o Assign an appropriate methodology for the project.

o Best practices and discipline for development (code-review, incorporate unit testing into work, source management, etc.)

o Have students to present their solution approaches and work prior to implementation.

o Supply internal project management from faculty. Even if the engaging customer has their own project management, an internal faculty project manager should closely monitor student’s activity and progress.

o Be price-competitive. Between engagement fees and student enrollment, the cost of running the program should be well taken care of. Be as cost competitive with off-shore competition as possible.

o Encourage/sponsor government legislation to promote the use of Off-Shore On-Campus programs in the form of tax-breaks to organizations engaging in this form of resource utilization over off-shore. It certainly will help the university win in competitive contracts.

o The quality of the program will be measured by industry by the results obtained, how the program is managed by faculty, and repeat business.

Intermix with Industry

Most academic institutions make use of industry professionals on advisory boards. These are most certainly effective. While advisory boards serve a valuable purpose in exposing the institution to industry trends and needs, there is an element that can be further pursued.

When an advisory board meets, set aside time for the industry representatives to individually meet directly with students. This can be in the form of an informal doughnuts or pizza get-together or place the industry professional in a given classroom lecture to speak with the students and answer questions.

The value here is enormous. Students get opportunity to talk with future employers gaining first-hand direction and insights. Industry professionals can instill the values, expectations and industry needs.

Likewise, many European universities require their faculty to engage with industry – mainly on a consulting basis. This keeps them first-hand in touch with how industry operates and the demands and expectations are being placed on resources.

Faculty either participating with industry directly or perhaps in the form of Off-Shore On-Campus will carry a better connection with industry.

Industry/Vendor Taught Seminars

Industry developers are often encouraged by their employers to attend several seminars each year relating to the technologies with which they are involved. Both faculty and students can benefit from this type of exposure.

One Music class I took required attendance at 6 external musical performances in addition to normal coursework to pass the class. The same benefits of external exposure to technology await a student of Computer Science.

Technology providers often provide free seminars on the technologies they offer. Request technology providers to bring seminars into your institution. Many organizations and companies have requirements for their employees to teach seminars – seek these out and utilize them. Technological user groups often bring in guest speakers.

The purpose is beyond exposure to industry and new technologies, it also promotes the fact that after graduation, learning never stops.

Bi-Weekly Internal Seminars Encouraged

A bi-weekly series of internally led seminars will benefit both faculty and students from various perspectives. Faculty can present research and findings. Senior level students should be strongly encouraged to present at least once a year either individually or as part of a team. This will further enhance presentation skills and expose them to defending their ideas.

Invite industry professionals to participate in these sessions as they can be utilized as a sounding board.

Note:

Many students today will be the industry innovators of tomorrow. It is these very students that you'll want to return to lead these seminars.

Stay Visible in Industry

Many product manufacturers and corporations welcome advice from academia. Make sure you are visible to them by attending their seminars and inviting them to present at your offerings. Encourage faculty to acquire time with industry in the form of consulting or assisting in research.

Making industry involvement a Key Performance Indicator (KPI) for faculty will benefit both the university and the individual – although you may run into some initial push-back. One university I am aware of is lowering the number of required teaching credit hours for incoming faculty making it a requirement that they either apply for a grant or participate in some form of industry engagement exposure with the additional time. I applaud this.

Managing an Off-Shore On-Campus project team offers great involvement with industry while promoting student involvement.

Final Thoughts

Why be so passionate about our education of future developers?

The answer comes from an industry expert, Keith Brophy, who states, "The ability of the well-educated, leading edge technology savvy fresh graduate to contribute is immense, and such a person can dive right into the fray, whereas the less well educated counterpart is rendered obsolete before the starting gate and may never catch up."

The appeal is simple – Focus on and bring science ideals into Computer Science. Utilize a new model with education being accountable to industry. Instill and nurture greater communications skills from the start.

It's not about the challenging task to move educational practices at the pace of technology. That would only turn educational institutions into skill-schools. Rather, it’s about modifying the educational model to supersede industry. Putting education at the forefront of technology is only possible by placing science into the equation and operating it as a science.

Utilizing industry professionals to supplement your efforts may be the key to accomplishing these goals. Exposing both the academic institution and its students to industry using the Off-Shore On-Campus model is a win-win situation for everyone involved.

Sticks & Stones

Jim Wilt — Mon, 27 Apr 2009 19:52:00 GMT

Its been many years since the first grade when Craig, the school bully, punched me, the new wimpiest kid, in the stomach on the way home from school. I can only guess Craig's motivation was that I was an easy target which would offer no resistance. I don't know if it was the physical pain or shear embarrassment, as it happened in front of all my classmates, that made me cry as I ran home. I was too embarrassed to tell any teachers the next day, but a classmate, Pam, did on my behalf and Craig received a wack from the principal's paddle, detention, and the wrath from my 3 years older brother as a result.

Only recently have I ever felt as under such unprovoked an attack as that day.

First, let me start by offering that I really like where Solutions & Infrastructure Architecture is going. As it matures and The Open Group, IASA, Microsoft, etc. begin to align with their respective definitions, I feel we are making great progress in creating a true profession. These are the ones that talk about the Seven Competencies: Leadership, Communication, Organizational Dynamics, Strategy, Process/Tactics, Technical Breadth, and Technical Depth. Mastery in these areas leads to a title of Solutions or Infrastructure Architect. Industry experience of around 15 years on average gets you here.

Next, let's recognize those day-to-day, in the code, SDLC, and masters of process architects who for lack of a better definition, Don Browning (a most highly respected colleague) calls Application Architect at his organization in Atlanta. Others have been known to label them as "Blue Collar" architects but I think this label is less respectful than Application Architect. I would guess these individuals outnumber S & I Architects by 20 to 100:1 and they carry 5-10 years experience.

Since the fall of the economy, I have seen a strong and continuous attack against the S & I Architects in favor of promoting attention to Application Architects. The attacks themselves generally take on the nature of bashing S & I to promote Application Architects:

"Ivory Tower architects are too far from code to know how to deliver a solution"
"Anyone who doesn't code every day is just an Enterprise Architect"
"Real Architects live and breathe in Visual Studio, not Visio or PowerPoint"

Interestingly, this attack is not coming from Application Architects (ironically, I see the S & I and Application Architects, themselves, maintaining a high respect for each other). Rather, the bully is senior/executive management coming from all industries across the board (manufacturing, financial, entertainment, technology, etc.) and from all sized organizations (12 employees to Fortune 100).

I am at a total loss for the behavioral bully aspect, but I think I can surmise that the financial crunch we all are under is driving us to consider lessoning our dependency on more expensive resources and trying to get more from less expensive and more readily available resources.

I think the warning Dave Taylor gave in his blog, Circuit City downgrades employees and guarantees its eventual demise, not only was an accurate prediction which came to fruition, but it might parallel what industry may experience when it forces Application Architects into elements for which they are not yet prepared, and removes those entities (i.e., S & I Architects) who willing want to mentor them. In the Circuit City case, you can be quite certain the less experienced workers made their best efforts with the best of intentions to deliver at the level of depth and quality of those they replaced, but they simply were not able to and are by no means to be blamed for the company's demise. It simply was a "management" decision (gamble?).

I totally respect that the current state of our our economy has made everything about "the bottom line" and I believe there are some really tough decisions being made (we've experienced a 20% reduction in the little strategy boutique I'm part of). These sometimes foster a reaction that is not always complimentary.

To this I simply say, "Sticks & Stones" and let the sometimes brash comments roll off the shoulder. When you really think about it, the comments one makes about others often says more about the author of the comments than the target of the comments.

To this effect, I never had to raise any rebuttal to the pummeling Craig gave me (he did apologize). The damage to himself his own actions invoked was far worse than the momentary pain he caused me.

So, industry, vent, release, and blame away. We S & I Architects have broad shoulders, strong backs, and can even take a pummeling now and then. We understand and we can help. Here's how:

Organizations who are focusing on strategy and planning by making a longer term investment into the bemused Ivory Tower architectural considerations may end up a step ahead as they are proactively focusing their energy on running the business of their business through strategic planning and technology alignment while working with their Application Architects to optimize/realize the implementation of their strategy.

Want to jump on board? Contact your Solutions or Infrastructure Architect who will be most willing to share and help!

What knowledge, skills, and experiences do graduates need to be competitive?

Jim Wilt — Mon, 30 Mar 2009 19:17:00 GMT

I'm bummed.

This Saturday is my semi-annual Central Michigan University College of Science & Technology Advisory Board meeting and I'll be returning from a business trip and have to miss it. One reason beyond getting to meet with some of the greatest scientific minds of our time is that we have a really cool agenda. The focus of this posting is:

6. Discussion – What knowledge, skills, and experiences do our graduates need to be competitive? To get those neurons firing…

Assuming that the core knowledge is in the major, what might be missing? What skills are needed for success? Statistics, data analysis, communication, comfort with international communication and travel? Finally, what experiences should our students have? Do they need internships, co-ops, summer jobs, research projects?

I didn't have to think long at all on this one as it comes up frequently among my colleagues (whom I hope post their opinions as well). Here's a quote from a recent gathering of world-wide colleagues when we were discussing new-hires: .

"Back in the 80's, new-hires worked 80-90 hour weeks on their own, thinking they were making a difference, and they did. Today, new hires feel they are owed the privilege of working 32 hour weeks and lack both the passion and work ethic to make a difference."

Discussion led to some possible reasons why:

Four (4) day school weeks and student/parent centered (not education centered) posturing of universities tend promote a lazy research/work ethic and will often set student's expectations as such
Many new hires are too often content with mediocrity. Perhaps employers need to create a reward system that capitalizes off the new "gamer" mentality that prevails in many technology new-hires (e.g., each successful project results in "moving-up to the next dev level with 5 virtual quality tokens")

My opinion:

Universities need to choose who they want to be:

Vocation training institutions who's metric of success is based on the quantity of job placements
Higher education institutions who's metric of success is based on the quality of their graduates' ability to excel at thinking and learning

Universities can excel at one or the other, but too often fail when they attempt both.

Skills which may make a difference in industry:

Communication (written, interpersonal, presentation)
Problem Solving (those that tend to come from mathematics and the pure sciences)
Learning to Accept Failure (gracefully) and to Learn from Failure
Internships and travel abroad can be most beneficial and a foreign language will take you even further
Learning to quickly learn and come up to speed on new concepts
Oral Exams to promote Thinking on your Feet will go a long way
If there is no passion, encourage students to instead pursue that for which they have a passion

Consider my Diagram of Success:

No matter where you set your sights, gravity (reality) takes you one step lower.

If you enter higher education with the goal of becoming rich, you will never be rich, satisfied, or content; you may eventually consider your life is in the toilet.
If you set your passion and goals to reaching some form of perfection (the star) in your chosen profession, you will never reach that perfection (and at times, this will frustrated you), but you will always seem to have enough finances to get by and you will find contentment and reward in your efforts. You may even have a major impact on industry.

Fun before Friction

Jim Wilt — Mon, 09 Feb 2009 12:44:00 GMT

I realize that I promised that I'd next be posting on treating complexity as a calculation similar to friction (i.e., the coefficient of complexity) - but I've shared my initial thoughts with greater minds than mine (Roger Sessions & Miha) and need more thought on the idea before I take it to pen.

In the meantime, a trusted colleague and friend, Peg McNicol, shared a book with me that while intended for the coffee table, really hit home for insightful thoughts around IT architecture.

The book, titled, "101 Things I Learned in Arcitecure School" by Matthew Frederick is solely intended to recap those things he learned while becoming a building architect - yet there are some many parallels to IT architecture. I'm compelled to share several of those thoughts here:

#14 Architecture begins with an idea. Good design solutions are not merely physically interesting but are driven by underlying ideas. An idea is a specific mental structure by which we organize, understand and give meaning to external experiences and information...

#21 An architect knows something about everything. An engineer knows everything about one thing. An architect is a generalist, not a specialist...

#28 A good designer isn't afraid to throw away a good idea. Just because an interesting idea occurs to you doesn't mean it belongs in the building you are designing. Subject every idea, brainstorm, random musing, and helpful suggestion to careful, critical consideration. Your goal as a designer should be to create an integrated whole, not to incorporate all the best features in your building whether or not they work together.

#29 Being process-oriented, not product-driven, is the most important and difficult skill for a designer to develop. [nuff-said there!]

#48 If you can't explain your ideas to your grandmother in terms she understands, you don't know your subject well enough. Some architects... use overly complex (and often meaningless!) language in an attempt to gain recognition and respect...

#81 Properly gaining control of the design process tends to feel like one is losing control of the design process. ...Being genuinely creative means that you don't know where you are going, even though you are responsible for shepherding the process. This requires something different from conventional, authoritarian control; a loose velvet tether is more likely to help. Engate the design process with patience... Don't try to solve a complex building in one week. Accept uncertainty... Don't seek to relieve your anxiety by marrying yourself prematurely to a design solution...

#86 Manage your ego. ...forget about what you want the building to be; instead ask, "What does the building want to be?"...

#92 "Always design a thing by considering it in its next larger context - a chair in a room, a room in a house, a house in an environment, an environment in a city plan." -- Eliel Saarinen

#97 Limitations encourage creativity.

#98 The Chinese symbol for crisis is comprised of two characters: one indicating "danger," the other, "opportunity." A design problem is not something to be overcome, but an opportunity to be embraced...

#101 Architects are late bloomers. Most architects do not hit their professional stride until around age 50! ...

Simply profound!

Thanks for sharing, Peg!

The ABCs of Unknowns

Jim Wilt — Sun, 14 Dec 2008 02:58:00 GMT

From my last post, I offered that looking at IT architecture through Physics Goggles provides a means of relating the measured realities for solution delivery to the complexities and unknowns involved.

In this post, I will extend the properties of solving physics problems to dealing with the many unknowns in solution delivery.

My esteemed colleague, Erik Johnson, points out the Donald Rumsfeld quote around the three types of unknowns. This is a perfect quote to start this thought exploration as it creates categories of "unknowns" to explore:

A - Unknowns we don't care about

B - Unknowns we must care about

C - Avoiding Solution Development Singularities

"...as we know, there are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns - - the ones we don't know we don't know."

- Donald Rumsfeld

Let's examine each of these individually. For illustration purposes, we'll base our discussion on the example of a falling penny discussed at aerospaceweb.org based on the physics of motion as first discovered by Galileo and Sir Isaac Newton.

A - Unknowns we don't care about

For this category, we'll use the example of dropping a penny from a tall building on the moon. The reason we are beginning on the moon is so that we have no opposing forces acting on our penny other than the moon's gravity.

The kinematic equitation of motion we use is:

To calculate the time it will take our penny to fall we solve this equation for t:

where

d = the height of our building

a = the acceleration due to gravity on the moon

The significance of this example is that while there theoretically are forces of drag acting on our penny, on the moon they are so negligible, we essentially can ignore them.

This defines our first category, unknowns we don't care about. The architect needs to rely on their experience and knowledge (e.g., of patterns) to discern which forces that may theoretically work against their solution are to be considered negligible and thus ignored.

The circumstances when this comes into play are often found in a session with stakeholders during what I like to call What-If Triage. This is that time when stakeholders present their what-if and how-do-I scenarios. Many unknowns can be articulated during these sessions and it is imperative that great care is taken to not damage delicate egos while placing aside identified unknowns we don't care about list.

The caution is to avoid influence from that strong opinioned stakeholder who attempts to force you to take into account forces that misalign the solution from the business strategy by introducing their own short term success factors that add no value.

A best practice for these unknowns is to identify them quickly and don't let them influence your solution.

This may seem similar to Roger Sessions' Second Law of Simplification (Any functionality that can be removed from an ABC should be removed), however, it is different in that eliminating unknowns does not necessarily eliminate functionality.

B - Unknowns we must care about

For this category, we'll continue the example of dropping a penny from a tall building, but this time it's on the earth. Since the earth has a significantly thicker atmosphere than the moon, you can be certain there will be significant forces working against gravity which we by no means can ignore.

This counter force is called drag which is due to friction from the earth's atmosphere. It is dependent on the velocity of the penny as it falls. Drag increases the faster the penny travels. As the penny is accelerated by gravity and the velocity increases, its drag also increases, but only to a point in which its terminal velocity is reached causing the penny's acceleration to become zero.

This is a completely different behavior from our penny falling on the moon!

The unknowns we could ignore earlier are now necessary to calculate the force of Drag:

where

D = drag force

ρ = air density

V = velocity

Sref = reference area

CD = coefficient of drag

Now, the terminal velocity is the point where the drag that is pushing against the falling penny's motion becomes equal to the weight of the penny (which is the mass, m, of the penny multiplied by the acceleration, g, from gravity). Thus, we can determine the terminal velocity to be:

It's not so simple anymore, is it? The penny on the earth actually falls 87% slower than the penny on the moon.

This defines our next category, unknowns we must care about. Nasty as these details are, we absolutely must address all of the unknowns or risk being at best, 87% wrong.

The following graph depicts how these unknowns dramatically affect both timeline and outcome:

The circumstances when this comes into play are often found in integration solutions where two or more systems attempt to share data with each other. When sharing medical information, for example, there are fields and formatting standards defined by HIPAA and HL7 that must be strictly adhered to.

Trouble begins when one system simply does not contain required data for the integration to occur. You cannot ignore it and you may not be allowed to artificially create it. Now you're facing significant timeline additions and changes to legacy systems for which you may not even have the expertise to address.

These are the thorniest of situations in which an architect must communicate issues of unknowns as early as possible to weigh the effort to resolve against the business strategy projected gains to justify the necessary added expense.

The architect needs to quantify:

o The time required to identify, educate themselves on, and factor in all the required unknowns

o An escalation path for the times necessary unknowns cannot be determined or resolved (this can be due to education or dependencies on other parties)

o Determining criteria for identifying runaway situations - those times unknowns feed off of each other to compound their negative impact

o Necessary education for stakeholder understanding of the forces at play

C - Avoiding Solution Development Singularities

Now let us consider dropping a penny on the earth, but this time, substitute the tall building with dropping it from outer space.

This introduces many, many more unknowns. We have to now consider things like the air density, ρ, will change based on altitude and the reference area of the penny, Sref, is also variable as the penny tumbles in the atmosphere. Now, lets add the force of Lift based on the Bernoulli Principle due to the aerodynamics of the penny and any shape deformation due to heat generated by Friction from the mentioned forces at play and you can see we are fast approaching more unknowns than we care to deal with.

Back in my aerospace/machine tool days, life was much simpler. As long as we worked with no more than five (5) degrees of freedom, we did not have to worry about singularities.

But wait, we worked with 13 degrees of freedom, so I guess life really wasn't any simpler. In fact we feared the thought of our algorithms creating a singularity as that meant snapping of limbs and costly damage.

A singularity in machine tool control is when there exists more than one single combination of axis positions to set the desired positioning of an end effector

We did have a solution, however. Knowing we could solve our forward kinematics w/o singularities at 5 degrees of freedom, our look-ahead logic would intelligently choose the most important 5 axes for movement in the next 20 ms block of time and lock the remaining 8 axes.

This sets the stage for our last category, avoiding solution development singularities. I offer that a solution development singularity be defined as both specific unknowns and the number of unknowns in which the team can no longer operate with continuous forward motion.

Specific unknowns that halt forward motion are easy, however, the number of unknowns that halt forward motion will depend on the complexity of the solution and the problem solving skills of the team. So, the number selected will be an architect's judgment of the solution space, their team, and their own leadership skills.

Once the architect identifies what constitutes a solution development singularity situation, they must then intelligently select which unknowns are the most important for the given iteration (using their experience and knowledge of the business strategy as their look-ahead logic). Then, they must lock out any other unknowns by making intelligent approximations/assumptions about them.

Consider, as an example, the development of a web site on a platform framework using a Software as a Service (SaaS) model where the server host organization supports only outgoing email. This leaves development of inbound email features, unknown.

Before the solution is released into production, it will obviously have to be moved to a hosting organization that supports bidirectional email. Until that time, the team must operate as if that unknown will at some point be resolved. At the time the hosting organization switch is made, the architect will lock other development variables until the inbound email features are vetted and brought into the solution's operational parameters. The experienced architect will also anticipate other unknowns to be introduced simply due to the switch of hosting organizations.

Sounds simple, doesn't it? We have all kinds of tools to help us quantify and track solution dynamics from project schedules to burn-down lists, and yet we still fail to manage our abundance of unknowns!

Because it is so important to prevent project meltdowns, the architect must monitor the team dynamics and work closely with project management to prioritize and lock-out unknowns to avoid solution development singularities. It is always a good practice to record locked-out unknowns with reminders so they don't become forgotten and burn you later because they were never dealt with. I find that by mentioning them at weekly project review meetings, they usually don't catch anyone by surprise when they are re-activated.

Recap/Summary

An architect can deal with the many solution unknowns in the following ways:

o Ignore unknowns that are not germane to the solution space, no matter how important they are made out to be.

o Never ignore unknowns that are critical to the solution no matter how nasty and complicated they are; be responsible in reporting them back to stakeholders to align their expectations to the timeline and cost adjustments, and establish appropriate escalation paths.

o Solution Development Singularities will shut a project down; prioritize unknowns and limit their number to a manageable size locking away less important unknowns with your best look-ahead logic.

In my next post, I will equate solution complexity with the force of friction defining what I call the Coefficient of Complexity.

When 60% is an "A"

Jim Wilt — Fri, 28 Nov 2008 08:45:00 GMT

At the recent Strategic Architecture Forum (SAF) in San Francisco, I was given opportunity to discuss the topic of "What an Architect Needs to Know"

My opening slides contained some statistics of what industry research organizations believe we should know:

Roger Sessions recently posted about Senate Bill S.3384 and Public Sector IT identifying how government through committee oversight can "fix" this. Roger accurately identifies, "There is no possible way that S.3384 can be implemented successfully unless the government first takes steps to understand and manage IT complexity."

A key point Roger captures here is complexity. It seems to me that many organizations prefer to adopt a philosophy around Occam's Razor thinking they will have greater success if they pursue simplicity and eliminate complexity. There obviously is some merit with this philosophy except when the problem space/solution truly is complex.

The government suggests the best approach to deal with complexity is to manage it up front and halt the development process once there is a 20-40% deviation for the committee to evaluate the situation. The idea we can mitigate the effects of complexity by unraveling it all up front implies, to me at least, the problem space/solution was never complex to begin with.

In my talk, I offered a different way to look at all of this.

While pursing my physics degree, I was soberly awakened to the approach the Central Michigan University Physics Department uses to teach the art of dealing with complex problems.

When I would take a physics exam, 50% of the test covered material which we were exposed leading up to the exam, and 50% of the test was material we had never before seen. My professors were interested in not only knowing how much of the material I had learned, but more importantly, how did I apply that which I knew to that which I had never seen and did not know.

What this meant was that a 60% on a test could easily be an "A"

Is it just me, or does this closely align with the type of situations we IT Architects deal with in our problem space/solution deliveries? We are brought in to apply our previous experiences and knowledge (often in the form of proven patterns) to problems with many unknowns. While we can't guarantee success in terms of what outside observers use to measure success, we can apply our Deep Smarts to attack the problems presented using the best known techniques and practices to solve the unknown.

As the unknown becomes more known, our industry responds with frameworks and even packaged software solutions. But, this does not occur overnight or with the first iteration of a solution. It takes years for this to happen. At one time, for example, a spreadsheet was a completely new concept from a software perspective. Today, it is a commodity.

Deep Smarts are the engine of any organization as well as the essential value that individuals build throughout their careers. Distinct from IQ, this type of expertise consists of practical wisdom: accumulated knowledge, know-how, and intuition gained through extensive experience.

As we work with customers, we try to get a grasp for how well what they are seeking to accomplish with technology aligns with the business problem they seek to solve and if a package or framework exists that will expedite the solution. We then compare the knowns to the unknowns to suggest solution approaches that, on one end, will be more "out-of-box" but perhaps less aligned with business needs to the other end with more complex adaptations that offer the flexibility to more closely align to the business strategy and promote user adoption. Of course, projected timelines and costs equally span.

If we were to consider that approaching IT Solutions is more like a Physics Exam, we might conclude that the outside assessment of 60% is a more accurate representation of the problem space and not so much the solution process. When looking at solutions with more unknowns in this way, we see that having real "problem solvers" in the solution delivery team is as important to reliability & repeatability as is the proper choice of process, methodology, and if a framework or package applies. It is the resources that do the deep-thinking that's required to deal with the solution's unknowns.

We spend much time focusing on tools (processes, methodologies, frameworks, scheduling, etc.) to promote better alignment of technology with business strategy, but our outside observer metrics just don't seem to improve at a rate you might expect. Perhaps our focus is neglecting the inclusion and refinement of proper problem-solving resources with the technology and business strategy components.

A formula for better success? They must all work together!

Industry often diminishes the human factor by promoting stellar tools that seek to automate/ensure quality. I would suggest that industry research has clearly identified those alone are not enough and perhaps our smoking gun for the poor industry metrics is both not understanding what they really identify (a tough problem space) and the missing leg of the stable solution triad (problem-solving resources).

If organizations seek to better develop and utilize their individuals with Deep Smarts as an active role in their solution delivery, they might realize steeper gains in success metrics.

In my next posting, I will fall back to physics to contemplate approaches for problem-solvers to address problem spaces with many unknowns.

Solutions aren't solved by technology, they are solved by people.

Jim Wilt — Wed, 26 Nov 2008 22:53:00 GMT

Solutions aren't solved by technology, they are solved by people. Technology is just a tool people use.

When a solution fails and we blame technology, nobody wins. When a solution fails and people take responsibility to learn, adjust, and realign, we win. Therefore, seek the tool (business process, architecture, technology, etc.) which will best align with your business to empower people.