Sunday, December 4, 2011

How bad is your RPG Code

I've recently participated in a number of discussions that appear on various forums discussing RPG, its future, and value.  Frequently these discussions are led by highly skilled professional programmers who utilize the latest features and facilities of RPG and write good structured modular code leveraging procedures, service programs and advanced features of ILE.

I absolutely agree that the most current version of the RPG language has the capabilities of being a superb procedural language and can offer the best structure and results that one could expect from a procedural language.

After thinking about this and looking the realities of most IBM i shops, the simple reality is that most shops do not have programmers of the quality of the folks that write these arguments favoring RPG.  In a weird tribute to RPG, it is a language that can be learned, used, and implemented by people with very little training and a virtually no background in computers, technology, or systems.

In fact the vast majority of RPG programmers were blue collar workers without college degrees or training who learned RPG in in-house IBM sponsored training classes, Junior college courses, or 3rd party taught classes. 

The vast majority of programmers have never written an RPG program from scratch (i.e. an empty SEU or RDp file) but rather copy and modify existing programs, sample programs obtained from IBM redbooks,or articles in various books, magazines, and publications.  Few of these sample programs were examples of good code.

Over the years (and many of you have programs that were originally written 40 to 50 years ago and modified numerous times over the years written by people with all sorts of skills.

Cleaning or replacing this code base is the challenge faced by most IBM i customers.  Some vendors including IBM advocate code conversion to EGL (enterprise generation language) or Java.  There are many vendors who offer conversion to other language including various Windows technologies.

Straight conversion can cost as much as $1 per line of code resulting in project prices of several million dollars.  Conversion projects are extremely risky as the vendor will convert your code from RPG to another language, but they are not responsible for testing or insuring that the new language based system still works and performs well.

Most companies have hundreds of thousands of lines of redundant code, code that is never executed, or should otherwise be cleaned up and improved before any conversion is undertaken.

Some vendors (including IBM) advocate using their conversion tools to convert 5250 green screen applications to web based applications.  They argue that you can enhance the web presentation after the conversion. 

Vendors like IBM or the many others have superbly skilled programmers and architects.  They have no experience with corporate systems built and modified over 40 to 50 years and really have no idea what a company's system portfolio contains.  They would be shocked and horrified at that 12,000 line monster that drives one of your critical business applications with  the User Interface, business rules, and database access as well as control logic all embedded somewhere in the beast (along with commented out code from previous generations and code that can never be executed).

I recommend use of tools like Databorough's X-Analysis along with a rigorous methodology that mandates restructuring and modernization of each program as current projects require the program to be modified.  I also advocate a rigorous program of peer review carefully organized to insure that your very best programmers are reviewing others code. 

If you don't want to invest in X-Analysis, then use Hawkeye or at a minimum use DSPPGMREF to an outfile and write some programs to produce reports on the structure of your systems. 

Once again, it is only a matter of time until IBM drops support for IBM i and therefore RPG (which only runs on the IBM i operating system).  You have time to modernize and replace your RPG based systems but you need to start now.  When IBM does drop support for IBM i they will generally support it for 3 years after they announce that the product is discontinued.  Even then you can still run your systems and tools as long as you don't need to upgrade your machine or operating system. 

The key is to realize that IBM i and RPG will eventually be withdrawn and support discontinued by IBM.  They are offering their own tools to convert to Java and modernize now.  You need to take the path that makes most sense for you, but you need to move on this.

Friday, December 2, 2011

Will complacency kill IBM i?

While most westerners rejoiced the recent events of unrest and social reform in the middle east, i am sure many also asked "what took them so long?"

When I tried to answer the question myself about why a large group of intelligent, educated people would accept such leadership, and along with it a stranglehold on their innovation and freedom for so long, i realized that all communities fall into that trap at some stage. Its called complacency.

The IBM i community is no exception.

Think about it for a moment. there has been no real innovation our of IBM i that is directly relevant to the IBM i community since the 80's. in fact there have been some serious misdemeanors on their part. here's a couple: ILE - what's the point of that? if you going to go OO, go OO, don't just poke at it with a long stick! a friend of mine who worked on the RPG compiler team at that time told me that there was a real plan to go the whole way, and truly modernize RPG, but it was vetoed by management. here's another: webfacing - no further comment needed. what about WDSc? basically it destroyed your PC just to get a notepad style code editor up! no wonder people still use SEU. EGL? Great idea, even pretty good implementation for IBM, totally pointless in reality. Scenario: "Hey IBM i have 5 million lines of RPG, and aging work force. i love the platform and IBM. what should i do? answer: Rewrite your entire code base in a language no one uses, using a work force that doesn't exist. huh?

What exactly is a third party vendor? I have asked this question of many people, Mike Smith (ex IBM Chief Architect), Jon Paris, myself, Rich Hume, end users, developers, ISV's, in fact anyone i can ambush into listening to me. The answer it seems is: "not IBM"

So what IP grew the IBM i community to its current size? Firstly of course OS/400 and RPG. But from that moment on: MAPICS, BPICS, JDE, System 21, SYNON, LANSA, PRMS, essentially third party vendors. or rather "not IBM".

There are many in the IBM i community that would skip a system i news mag article about the science of business rules recovery of RPG systems with X-Analysis because it is labeled"Advertorial", in favor of 15 pages of how RDi miraculously helps them edit the one program they write per year these days. Similarly there are some who whinge to Jon Paris about third party vendors lowering the tone of his events and events like his.

the last few publications and events that remain as the conduits to the IBM i community, and who still support the "not IBM" mantra as a given are thankfully giving way to more viral and real communities such as we see on some linked in groups.

I don't think complacency will kill the IBM i community, but hopefully it will kill the bovine leaches who are too lazy to listen to anything but the PR drivel that mostly comes from IBM these days.

Monday, November 21, 2011

RPG Job Trend Stats

Here are two interesting graphs about the trend of RPG jobs. Whatever the implications are for the future of the IBM i, it certainly does provide little or no incentive for anyone to learn RPG.

Here are two interesting graphs about the trend of IBM i Jobs:


Thursday, February 10, 2011

How should we be developing applications for the IBM i today?

This is a response to a discussion started by Gary Lewis on Linked In - http://lnkd.in/EUFmNi

We have been working on this specific subject since 1998. Here’s a little historical and contextual framework:

While we have built RPG based re-engineering tools since 1987, in 1999 we started a project to develop tooling to completely re-engineer an RPG application into an event driven MVC model with a web/gui front end. Oddly enough this was inspired after we wrote a tool to put RPG into SYNON at the request of IBM way back then. The objective of the new project was not to just to technically achieve a modern GUI on as/400 based system, but to produce reworked code that was more maintainable than previously, and also adopted the most mainstream technologies for development and deployment.

It has been a long road that includes writing an IBM Redbook on the subject (click here) and various technology/development partnerships including Rational, Oracle, Microsoft, MyEclipseBlue, Fiserv, Mapics, SSA Global, and various other end user companies. We have built technologies for generating View Controller layers in Java Servlets and generated HTML, RCP on Eclipse. Java Swing fat client, JSON/Web2.0 dynamic web client, EGL JSP's, JSP and JSF using Spring/Hibernate, ASP, Silverlight, Flex, and more recently RPGLE. We have tried and tested almost all tactical technologies such as wefacing, emulation, special files, Dynamic Screen Mangement API’s, even CGi as part of the R&D effort. Also know we have assessed these approaches on many of the largest existing as/400 apps on the world, a good number of which are our customers.

Now let me share our findings and evolved best practices by answering your points more directly. These results are not just about technology fads or personal preferences, but the accumulation of literally 12 years of R&D specially on this subject, just so we can build sellable tools reengineer effectively into this new environment. I won’t elaborate on why with all points, but just to give you an example of lessons learned: What we discovered was that when building increasingly sophisticated UI’s as technologies and user requirements advanced, a critical mass was soon reached where the code used to render server generated screens became too complicated to maintain productively. Turn it on it’s head and build the view and Controller in a modern language(using event driven languages tightly integrated with screen actions like VB), and ongoing maintenance and development was much simpler and could be deskilled significantly for typical business apps. The reason target IDE & Technology was a big part of the R&D is that what made the as/400 big in the first place: The development ecosystem productivity PLUS the reliability, scalability etc. So any company that would potentially buy our re-engineering tooling would have that expectation built into the requirement.

Gary's questions included:

How would the application architecture look like?

Model View Controller – VC in any modern UI language, Model in RPGLE-Free, and DB2/i for all the good stuff

Should we just use the IBM i as a database?
no. Business logic/Model layer too

Should business rules be in the DB, stored procedures or application logic?
Application Logic – this can be implemented and used as stored procedures simultaneously. Implementing in the DB works for pure constraints, but the dev tooling and process is slowed down by over engineering where you put business logic

Where should the application logic reside?
User driven and system, Navigational logic in the Controller, Conditional navigation logic in the Model – This should not always be strictly adhered to in all instances.

What language and IDE would you use?
VC : VS2010, MyEclipseBlue, and the one that has impressed us the most in 12 years: Oracle ADF
Model: RDp

DB design - Should we use surrogate keys?
Only on new databases where possible.