How should we be developing applications for the IBM i today?

This is a response to a discussion started by Gary Lewis on Linked In - http://lnkd.in/EUFmNi

We have been working on this specific subject since 1998. Here’s a little historical and contextual framework:

While we have built RPG based re-engineering tools since 1987, in 1999 we started a project to develop tooling to completely re-engineer an RPG application into an event driven MVC model with a web/gui front end. Oddly enough this was inspired after we wrote a tool to put RPG into SYNON at the request of IBM way back then. The objective of the new project was not to just to technically achieve a modern GUI on as/400 based system, but to produce reworked code that was more maintainable than previously, and also adopted the most mainstream technologies for development and deployment.

It has been a long road that includes writing an IBM Redbook on the subject (click here) and various technology/development partnerships including Rational, Oracle, Microsoft, MyEclipseBlue, Fiserv, Mapics, SSA Global, and various other end user companies. We have built technologies for generating View Controller layers in Java Servlets and generated HTML, RCP on Eclipse. Java Swing fat client, JSON/Web2.0 dynamic web client, EGL JSP's, JSP and JSF using Spring/Hibernate, ASP, Silverlight, Flex, and more recently RPGLE. We have tried and tested almost all tactical technologies such as wefacing, emulation, special files, Dynamic Screen Mangement API’s, even CGi as part of the R&D effort. Also know we have assessed these approaches on many of the largest existing as/400 apps on the world, a good number of which are our customers.

Now let me share our findings and evolved best practices by answering your points more directly. These results are not just about technology fads or personal preferences, but the accumulation of literally 12 years of R&D specially on this subject, just so we can build sellable tools reengineer effectively into this new environment. I won’t elaborate on why with all points, but just to give you an example of lessons learned: What we discovered was that when building increasingly sophisticated UI’s as technologies and user requirements advanced, a critical mass was soon reached where the code used to render server generated screens became too complicated to maintain productively. Turn it on it’s head and build the view and Controller in a modern language(using event driven languages tightly integrated with screen actions like VB), and ongoing maintenance and development was much simpler and could be deskilled significantly for typical business apps. The reason target IDE & Technology was a big part of the R&D is that what made the as/400 big in the first place: The development ecosystem productivity PLUS the reliability, scalability etc. So any company that would potentially buy our re-engineering tooling would have that expectation built into the requirement.

Gary's questions included:

How would the application architecture look like?

Model View Controller – VC in any modern UI language, Model in RPGLE-Free, and DB2/i for all the good stuff

Should we just use the IBM i as a database?
no. Business logic/Model layer too

Should business rules be in the DB, stored procedures or application logic?
Application Logic – this can be implemented and used as stored procedures simultaneously. Implementing in the DB works for pure constraints, but the dev tooling and process is slowed down by over engineering where you put business logic

Where should the application logic reside?
User driven and system, Navigational logic in the Controller, Conditional navigation logic in the Model – This should not always be strictly adhered to in all instances.

What language and IDE would you use?
VC : VS2010, MyEclipseBlue, and the one that has impressed us the most in 12 years: Oracle ADF
Model: RDp

DB design - Should we use surrogate keys?
Only on new databases where possible.

RPG Free or Fixed? OR Modernize your code base

As most of you who read this blog know I am no fan of RPG.  I in fact urge you to create plans to move away from RPG as fast as is feasible for your business.  I also recognize that this cannot be done overnight and that any realistic plan is really a 7 to 10 year plan.

I'm writing this blog tonight after a long hiatus after reading a somewhat disturbing discussion on one of the Linkedin forums about whether or not folks should using free form RPG.

While the creators and contributors were well intentioned, they are really missing the key point.  Namely the need to restructure, and modernize their existing application systems into modular ILE base reusable modular code.

I would personally take a close look at Databorough LTD's X-Analysis and related suite of products as a first step in discovery, analysis, simplification, and modernization.  I would then look at tools to refactor code into modular resuable components that can be leverage in web services and other modern technologies.  This initiative which can be highly automated can greatly reduce the cost, complexity, and risk of moving away from RPG and eliminating your dependency on IBM i.

It can also enable you to leverage your existing systems and modern service based applications and provide significant value to your organization.

The Internet is Running out of IP Addresses

According to the folks at http://bit.ly/6QK89F we will run out of IP4 Internet addresses on Oct 25, 2012.

This is so typical of the IT industry.  Back in 1995 when the Internet was opened for commercial use work was well underway to support IPV6 and most hardware and software companies including IBM with the iSeries have had support for IPV6 for years.  This is pristine software as virtually NO ONE has ever used it! 

Oh well, the world needs a new crisis!  Maybe Al Gore who as you will recall invented the Internet will step up to the plate and fix this!  Oh yes, of course those other companies will change their addressing schemes to IPV6 and we will still be able to use IPV4! 

Right! 

Well folks this one will be interesting! 

Databorough Announces X-Analysis Lite

X-Analysis Lite offers the industry’s most powerful cross referencing at budget prices

Toronto, Canada -- Thursday, 03 December 2009, -- Databorough, a leading vendor of tools for IBM i, today announced a cut-down version of X-Analysis its flagship product, the recognized world leader in analysis, documentation, design recovery and reengineering tools for system i.

X-Analysis Lite is Databorough’s response to the demand from the broad community of smaller IBM i shops that love the X-Analysis functionality, but don’t have budgets that stretch to the enterprise versions of the product. Along with a set of proven and powerful analysis features, the Lite package includes three of the most popular and productive features from the enterprise versions of X-Analysis. They are: Application Mapping, Variable Tracing Technology, and Interactive Graphical Diagramming, and provide critical functionality to thousands of RPG and COBOL developers using X-Analysis 8 on IBM i worldwide. X-Analysis Lite also comes with bi-directional interfaces to IBM’s WDSc and RDi products, as well as stand-alone versions for traditional SEU users. An introduction discount for upgrades from Hawkeye is available.

IBM is now hosting a configuration of the Lite on the IBM Enterprise Modernization Sandbox for i, see:

IBM Enterprise Modernization for System i: Discover and understand RPG applications with X-Analysis

Application Mapping builds a complete map of an entire application and stores it in a readable DB2/400 database library. All objects are cross referenced first, and then every single source member of any type is parsed and mapped down to individual variables and how they relate to other variables, not just fields. A number of X-Analysis users, ISV’s, and partners have already developed their own add-ons and services using the Application Map Repository.

Variable Tracing Technology uses the Application Map to programmatically trace variables and fields across and entire system. This means that impact analysis for changes can be done for multiple levels, or variants of a field or variable, in a single inquiry. This unique to X-Analysis feature, has saved many companies thousands of man hours in analysis time. See our Case Studies for details. It is also one of the key features that prompts users of legacy tools such as Hawkeye to upgrade.

Interactive Graphical Diagramming is the most commonly used and popular feature of the X-Analysis suite. Developers, analysts and support teams can see structure and process in interactive, graphical color-coded diagrams. Drill down capability into color coded source browsing and navigation through diagrams across the entire application. “GPS for Legacy Applications!” as one user described it.

Running over RPGII/III/IV/Free, CL and COBOL on IBM i X-Analysis Lite is available direct from Databorough or through our extensive partner network.

Additional information is available on the Web at http://www.databorough.com/lite

About Databorough

Databorough Ltd. is a leading vendor of innovative software tools enhancing IBM i legacy applications and taking them into a 21^st century environment.

Databorough’s software is developed and maintained by development teams in Canada, UK, France and India. It is at work, worldwide, in companies including market leaders in the manufacturing, distribution, retail, services and financial and application software sectors.

Databorough is headquartered in Weybridge, near London, United Kingdom. The company’s main North American office is in Toronto, Canada and its main R&D centre is in Lucknow, India.

###

All product and company names may be trademarks or registered trademarks of their respective holders.

CONTACTS:

Databorough:
Stuart Milligan, Tel: (917) 267-7523
email: stuartm@databorough.com

How much RPG application code is still relevant & unique

This is the most important question that many CIO's who have as/400 applications are asking themselves. Why are they asking this?

Cost and risk. It costs a lot more money to write your own applications in house than it does generally to buy one off the shelf. The reason being that you have to hire people to write and maintain the code. That's where the risk comes in. If i lose the person who wrote the code then i also potentially impact my ability to enhance and support that same code at the same costs and quality.

Since the introduction of the as/400 in 1988 a lot of custom code has been written on the platform in RPG. The fact that the majority of this code is still around and in use, is testament toRPG itself which makes it possible for analysts with business knowledge, to write a large portion of the code.

In helping customers worldwide to analyse and re-engineer their RPG applications over the last 25 years Databorough has accrued a fairly unique knowledge of the size and characteristics of the this code base in general. we estimate that there is at least one billion lines of custom code in existence on as/400's worldwide.

Before any CIO can make an informed decision on what application strategy to adopt: rewrite, modernize, replace, exactly how much of this code is still relevant and unique to the business must be known. This is in terms of business rules, data model design and flow/process logic. Extracting this information quantitatively, narrating it and documenting it on a million lines ofRPG code that has been written over 20 years, would take 5 people who would need excellentRPG and analysis skills, at least one year working full time.

That's at least half a million dollars direct cost. That is of course if there were enough such people available to do this work over and above ongoing support and maintenance demands on these same applications.

Over the last decade Databorough has been enhancing X-Analysis to the point where it can now automatically extract, narrate and document business rules, data models and process/flow logic for entire systems. Some of the systems X-Analysis runs over currently have in excess of 50 million lines of code, some as little as half a million. The difference in extraction time is usually a couple of hours.

The added benefit is that these extracted specifications can then be used to map to off-the shelf apps, as input specs for rewriting the unique and relevant parts, and even to automatically re-engineer the existing code where necessary.

This might make it clearer as to why IBM promote and use X-Analysis themselves in conjunction with their own tools, a few links to this are detailed below. There are also a few of our partners and independent agents who are offering modernization assessment services which help start this process. Some of these include, Sirius, ADC-AustinTech, IBM Global Services, and Oxford International.

So CIO's don't need to guess or accept others guesswork. They can make these critical decisions on provable, current, accurate and quantifiable information.

Here are a couple of useful links:

X-Analysis Design Recovery

IBM Redbook on X-Analysis

IBM Modernisation Strategy

Modernization Concepts Guide

Stuart

How much RPG code is Relevant?

Stuart started an excellent post on the value of a company's inventory of RPG. In my life before I went to work for IBM, I was CTO for a major insurance company who developed most of their systems in-house. We had a legacy of RPG code dating back over 40 years beginning with an IBM System III and RPG-II.

While we had some excellent RPG programmers over the years we also had many that were not so excellent and wrote code that worked but was quite frankly not very maintainable. This code would read like an English language conversation, had incredible amounts of redundant IO -- yes the same record read each time an edit, validation, or field was changed on a single screen.

The screen handling IO was embedded with business logic to the point that it was virtually impossible to segregate business logic form 5250 IO from Database access. In other words, spaghetti code.

As a company moves toward modern UI's and services like Web 2.0 and Web Services, it would be incredibly desirable to leverage existing business logic in existing RPG code. Extracting viable bits of code into reusable components can be a daunting task making a total rewrite of the application a better choice for many CIO's.

Databorough's X-Analysis offers some hope and tools to facilitate harvesting of business rules into reusable components (ILE modules, procedures, or at least subroutines). It also provides via its English Language psuedo code a way to facilitate a straight rewrite of code if that's the route you choose in the same or different language.

BTW, don't blame the programmers totally for the code you are stuck with. Much of the code you see in your shop are based on IBM examples from REDBOOKS or other "experts" BOOKS leveraging changing positions on performance over the years.

At one time you wanted your code totally sequential and in-line with no subroutines to optimize how the CPU would process your code.

Some of you may be stuck with code generated by 4GL's that is virtually unreadable. Quite often the code was generated using a 4GL vendors tools and your programmers then had to manually modify the code to add functionality that the 4GL did not provide. Once this was done you had code that could no longer be maintained by the 4GL tools. These programs have hundreds of lines of redundant unnecessary code.

The best solution are tools that can analyze and help you identify what is relevant and what is not.

What about RPG II code? I know that in my previous life a huge number of my RPG Programs were written in RPG II. Depending on your business requirements it may be worthwhile to first move your old RPG II or OPM RPG III code to ILE before doing anything else.

If you still have programmers writting in the old RPG II style you have some major skills problems to deal with as well as the code.

Databorough's tools are not a silver bullet, but they can definitely help you understand what you have and form a modernization strategy based on facts and provide CIO's with a means of managing a modernization project in an objective manner that minimizes risk.

X-Analysis 8 produce System Documents as PDF

PDF format is the defacto standard when it comes to sharing the documents. X-Analysis 8 can now produce System documentation as PDF.