Monday, November 5, 2012

How much has been "modernized?"

This was a question posted on a linked in forum recently.

"

How much has been "modernized?"

There's been a lot of talk for at least the past 20 years that systems would be "modernized." "We're going client-server & unplugging the mainframe." Still hasn't happened. Now c/s is legacy."

My assessment would be that very, very little has been modernized. We have the world leading analysis tool for the as/400 and of our thousands of users world-wide, less than 10% have done any real modernization. Most as/400 (and this is true of mainframe applications too) are large, complex beasts that are implemented for the most part using legacy languages and legacy architecture. Converting the language is not that big a problem, and automation tools have been around for decades to do this. but largely what they do is create legacy applications in modern languages. there are some real justifications for doing that on occasion, but not to reap the benefits of a modern application architecture, and development agility. 

To get a real modern application and the modern development benefits, you have to rewrite the legacy application. The problem with this (certainly on the as/400) is that the majority of legacy developers don't have the will, time, opportunity or capability to transition their own skills let alone a 10 million line mission critical application. Some have but not enough to create a pool of resource that would make any difference. 

Modern developers and often managers don't have the knowledge or experience of the business, and even if they did, the cost of rewriting manually is prohibitive and could take many, many years, and hugely risky. 

So the result is kicking the can down the road being the most prevalent attitude i see worldwide. Not that some management aren't interested in doing something, they just haven't had the means or method. 

To change this, automation is the only answer. Not automation by code conversion or even intelligent code conversion to produce neater legacy code in a modern language. Automation of reusable design recovery, complete transformation of the architecture and rewrite of the code. The drivers must be developers with a modern perspective, not legacy developers with modern skills, but a legacy perspective. so it should be a "harvest what we can from the legacy " rather than carefully preserving legacy in a modern language. The three aspects of automation that are required to effect this radical change are:

- Automation of complete system design recovery reusable designs/rules/models 
- Automated Transformation of architecture 
- Automated Generation of code 


The most productive part of automation is design recovery. It is also the part that has achieved the highest degree of automation(with our tools anyway). If functionally delivered in the correct way (graphical, interactive, drill down, models, flows, exports to UML/XML/DDL) it also solves another big inhibitor to modernization: putting the power of knowledge in the hands of new developers and management, and often legacy developers too (finally they can articulate what they have been doing for the last 25 years). 

Subsequent automation of transformation and code generation provides varying degrees of automation and cost benefits. But even modest levels of 50% can save 10’s of millions of dollars and years of risky effort. Projects done with our tooling have already delivered 80-95% automation of a manual rewrite. 

Most significantly we are seeing a release of pent-up frustration by companies who know the problem, know how it could be solved but couldn't find a technology that could deliver the solution with any real automation. 

I don’t believe that this will change the mentality of the many can kickers, who just want to see out their retirement in peace. It does mean that new and future developers/managers can harvest the value in their legacy systems, once the old guard has stepped down.

Sunday, December 4, 2011

How bad is your RPG Code

I've recently participated in a number of discussions that appear on various forums discussing RPG, its future, and value.  Frequently these discussions are led by highly skilled professional programmers who utilize the latest features and facilities of RPG and write good structured modular code leveraging procedures, service programs and advanced features of ILE.

I absolutely agree that the most current version of the RPG language has the capabilities of being a superb procedural language and can offer the best structure and results that one could expect from a procedural language.

After thinking about this and looking the realities of most IBM i shops, the simple reality is that most shops do not have programmers of the quality of the folks that write these arguments favoring RPG.  In a weird tribute to RPG, it is a language that can be learned, used, and implemented by people with very little training and a virtually no background in computers, technology, or systems.

In fact the vast majority of RPG programmers were blue collar workers without college degrees or training who learned RPG in in-house IBM sponsored training classes, Junior college courses, or 3rd party taught classes. 

The vast majority of programmers have never written an RPG program from scratch (i.e. an empty SEU or RDp file) but rather copy and modify existing programs, sample programs obtained from IBM redbooks,or articles in various books, magazines, and publications.  Few of these sample programs were examples of good code.

Over the years (and many of you have programs that were originally written 40 to 50 years ago and modified numerous times over the years written by people with all sorts of skills.

Cleaning or replacing this code base is the challenge faced by most IBM i customers.  Some vendors including IBM advocate code conversion to EGL (enterprise generation language) or Java.  There are many vendors who offer conversion to other language including various Windows technologies.

Straight conversion can cost as much as $1 per line of code resulting in project prices of several million dollars.  Conversion projects are extremely risky as the vendor will convert your code from RPG to another language, but they are not responsible for testing or insuring that the new language based system still works and performs well.

Most companies have hundreds of thousands of lines of redundant code, code that is never executed, or should otherwise be cleaned up and improved before any conversion is undertaken.

Some vendors (including IBM) advocate using their conversion tools to convert 5250 green screen applications to web based applications.  They argue that you can enhance the web presentation after the conversion. 

Vendors like IBM or the many others have superbly skilled programmers and architects.  They have no experience with corporate systems built and modified over 40 to 50 years and really have no idea what a company's system portfolio contains.  They would be shocked and horrified at that 12,000 line monster that drives one of your critical business applications with  the User Interface, business rules, and database access as well as control logic all embedded somewhere in the beast (along with commented out code from previous generations and code that can never be executed).

I recommend use of tools like Databorough's X-Analysis along with a rigorous methodology that mandates restructuring and modernization of each program as current projects require the program to be modified.  I also advocate a rigorous program of peer review carefully organized to insure that your very best programmers are reviewing others code. 

If you don't want to invest in X-Analysis, then use Hawkeye or at a minimum use DSPPGMREF to an outfile and write some programs to produce reports on the structure of your systems. 

Once again, it is only a matter of time until IBM drops support for IBM i and therefore RPG (which only runs on the IBM i operating system).  You have time to modernize and replace your RPG based systems but you need to start now.  When IBM does drop support for IBM i they will generally support it for 3 years after they announce that the product is discontinued.  Even then you can still run your systems and tools as long as you don't need to upgrade your machine or operating system. 

The key is to realize that IBM i and RPG will eventually be withdrawn and support discontinued by IBM.  They are offering their own tools to convert to Java and modernize now.  You need to take the path that makes most sense for you, but you need to move on this.

Friday, December 2, 2011

Will complacency kill IBM i?

While most westerners rejoiced the recent events of unrest and social reform in the middle east, i am sure many also asked "what took them so long?"

When I tried to answer the question myself about why a large group of intelligent, educated people would accept such leadership, and along with it a stranglehold on their innovation and freedom for so long, i realized that all communities fall into that trap at some stage. Its called complacency.

The IBM i community is no exception.

Think about it for a moment. there has been no real innovation our of IBM i that is directly relevant to the IBM i community since the 80's. in fact there have been some serious misdemeanors on their part. here's a couple: ILE - what's the point of that? if you going to go OO, go OO, don't just poke at it with a long stick! a friend of mine who worked on the RPG compiler team at that time told me that there was a real plan to go the whole way, and truly modernize RPG, but it was vetoed by management. here's another: webfacing - no further comment needed. what about WDSc? basically it destroyed your PC just to get a notepad style code editor up! no wonder people still use SEU. EGL? Great idea, even pretty good implementation for IBM, totally pointless in reality. Scenario: "Hey IBM i have 5 million lines of RPG, and aging work force. i love the platform and IBM. what should i do? answer: Rewrite your entire code base in a language no one uses, using a work force that doesn't exist. huh?

What exactly is a third party vendor? I have asked this question of many people, Mike Smith (ex IBM Chief Architect), Jon Paris, myself, Rich Hume, end users, developers, ISV's, in fact anyone i can ambush into listening to me. The answer it seems is: "not IBM"

So what IP grew the IBM i community to its current size? Firstly of course OS/400 and RPG. But from that moment on: MAPICS, BPICS, JDE, System 21, SYNON, LANSA, PRMS, essentially third party vendors. or rather "not IBM".

There are many in the IBM i community that would skip a system i news mag article about the science of business rules recovery of RPG systems with X-Analysis because it is labeled"Advertorial", in favor of 15 pages of how RDi miraculously helps them edit the one program they write per year these days. Similarly there are some who whinge to Jon Paris about third party vendors lowering the tone of his events and events like his.

the last few publications and events that remain as the conduits to the IBM i community, and who still support the "not IBM" mantra as a given are thankfully giving way to more viral and real communities such as we see on some linked in groups.

I don't think complacency will kill the IBM i community, but hopefully it will kill the bovine leaches who are too lazy to listen to anything but the PR drivel that mostly comes from IBM these days.

Monday, November 21, 2011

RPG Job Trend Stats

Here are two interesting graphs about the trend of RPG jobs. Whatever the implications are for the future of the IBM i, it certainly does provide little or no incentive for anyone to learn RPG.

Here are two interesting graphs about the trend of IBM i Jobs:


Thursday, February 10, 2011

How should we be developing applications for the IBM i today?

This is a response to a discussion started by Gary Lewis on Linked In - http://lnkd.in/EUFmNi

We have been working on this specific subject since 1998. Here’s a little historical and contextual framework:

While we have built RPG based re-engineering tools since 1987, in 1999 we started a project to develop tooling to completely re-engineer an RPG application into an event driven MVC model with a web/gui front end. Oddly enough this was inspired after we wrote a tool to put RPG into SYNON at the request of IBM way back then. The objective of the new project was not to just to technically achieve a modern GUI on as/400 based system, but to produce reworked code that was more maintainable than previously, and also adopted the most mainstream technologies for development and deployment.

It has been a long road that includes writing an IBM Redbook on the subject (click here) and various technology/development partnerships including Rational, Oracle, Microsoft, MyEclipseBlue, Fiserv, Mapics, SSA Global, and various other end user companies. We have built technologies for generating View Controller layers in Java Servlets and generated HTML, RCP on Eclipse. Java Swing fat client, JSON/Web2.0 dynamic web client, EGL JSP's, JSP and JSF using Spring/Hibernate, ASP, Silverlight, Flex, and more recently RPGLE. We have tried and tested almost all tactical technologies such as wefacing, emulation, special files, Dynamic Screen Mangement API’s, even CGi as part of the R&D effort. Also know we have assessed these approaches on many of the largest existing as/400 apps on the world, a good number of which are our customers.

Now let me share our findings and evolved best practices by answering your points more directly. These results are not just about technology fads or personal preferences, but the accumulation of literally 12 years of R&D specially on this subject, just so we can build sellable tools reengineer effectively into this new environment. I won’t elaborate on why with all points, but just to give you an example of lessons learned: What we discovered was that when building increasingly sophisticated UI’s as technologies and user requirements advanced, a critical mass was soon reached where the code used to render server generated screens became too complicated to maintain productively. Turn it on it’s head and build the view and Controller in a modern language(using event driven languages tightly integrated with screen actions like VB), and ongoing maintenance and development was much simpler and could be deskilled significantly for typical business apps. The reason target IDE & Technology was a big part of the R&D is that what made the as/400 big in the first place: The development ecosystem productivity PLUS the reliability, scalability etc. So any company that would potentially buy our re-engineering tooling would have that expectation built into the requirement.

Gary's questions included:

How would the application architecture look like?

Model View Controller – VC in any modern UI language, Model in RPGLE-Free, and DB2/i for all the good stuff

Should we just use the IBM i as a database?
no. Business logic/Model layer too

Should business rules be in the DB, stored procedures or application logic?
Application Logic – this can be implemented and used as stored procedures simultaneously. Implementing in the DB works for pure constraints, but the dev tooling and process is slowed down by over engineering where you put business logic

Where should the application logic reside?
User driven and system, Navigational logic in the Controller, Conditional navigation logic in the Model – This should not always be strictly adhered to in all instances.

What language and IDE would you use?
VC : VS2010, MyEclipseBlue, and the one that has impressed us the most in 12 years: Oracle ADF
Model: RDp

DB design - Should we use surrogate keys?
Only on new databases where possible.

Friday, March 26, 2010

RPG Free or Fixed? OR Modernize your code base

As most of you who read this blog know I am no fan of RPG.  I in fact urge you to create plans to move away from RPG as fast as is feasible for your business.  I also recognize that this cannot be done overnight and that any realistic plan is really a 7 to 10 year plan.

I'm writing this blog tonight after a long hiatus after reading a somewhat disturbing discussion on one of the Linkedin forums about whether or not folks should using free form RPG.

While the creators and contributors were well intentioned, they are really missing the key point.  Namely the need to restructure, and modernize their existing application systems into modular ILE base reusable modular code.

I would personally take a close look at Databorough LTD's X-Analysis and related suite of products as a first step in discovery, analysis, simplification, and modernization.  I would then look at tools to refactor code into modular resuable components that can be leverage in web services and other modern technologies.  This initiative which can be highly automated can greatly reduce the cost, complexity, and risk of moving away from RPG and eliminating your dependency on IBM i.

It can also enable you to leverage your existing systems and modern service based applications and provide significant value to your organization.