Reflecting on two years with Rails

Two years ago, I wrote this as part of preparation for a discussion on selecting a framework for web applications:

Rails Development tools: API docs, log viewers

Fluid (http:/fluidapp.com) is one of the coolest ideas I’ve come across for taking advantage of the web-service-as-application model that has been branded “Web 2.0.” It’s a tool for building site-specific browsers on Mac OS X; essentially, it creates small apps that bundle the webkit (a web browser in a UI widget for Mac OS that is the basis for Safari)  I recently came across an example of using Fluid that is particularly helpful for Rails developers, and could also be useful for work in other frameworks. Rails Jedi has pre-built Fluid applications for RailsBrain and NoobKit. Especially interesting is the RailsBrain app, because it works with an offline copy of the API docs.

Another useful app for developing Rails apps on Mac OS X is Spike, a tool for searching and analyzing Rails logs. Spike is one of those things that you don’t need until you really need it; I’ve used it so much, I find myself FTPing log files from our Linux test servers over to my laptop so that I can use Spike on them.

Tags: Rails, development, tools

Jon Stewart, who always has been only marginally funny, has for years been attempting to escape his comedic failures by following the likes of current-affairs ranters like Dennis Miller and Dennis Leary. Unfortunately, unlike Miller, Stewart is neither funny or insightful, and unlike Leary, Stewart is not named Dennis. Find a new schtick, Jon.

Rails config.gem vs. plugins

Ruby on Rails since v2.1(?) includes a mechanism for specifying rubygems that are required by the application. To use it, you place lines inside the Rails::Initializer.run section in config/environment.rb that indicate which versions of which gems are needed to run your Rails application.

Rails also includes some rake tasks that compliment this feature. There are tasks to install the gems, and others that will actually add installed gems to your project (in vendor/gems) - a feature that can make deployment a lot easier in some cases.

The mechanism isn’t perfect, however. The biggest issue with it is how it relates to Rails plugins. Often, a plugin will require one or more specific gems; in some cases, plugins are Rails-specific extensions to the gem (rails_rcov and selenium_on_rails come to mind.) Typically, the plugin author will document the requirement for the gem as part of the plugin documentation. In most cases, if the gem is not installed, your application will fail to start.

So, the obvious thing to do would be to identify those gems required by plugins and manage them via the Rails mechanisms, right? Herein lies the insidious problem: Plugin initializers run before Rails itself is initialized. They have to, so that they can add functionality to the Rails framework before the (many) singleton objects are created when Rails starts up. As a result, adding the gems to config.gem directives doesn’t insure that we won’t get errors related to missing gems.

In fact, if a plugin needs a gem we don’t have installed, it may prevent even the config.gem mechanism from running. In those cases, there is no choice but to install the gem before trying to initialize the Rails environment; that means that the gem has to be installed by something other than a task inside the Rails framework, or a task inside your application’s lib/tasks directory. Neither of those can be used because they aren’t available until the Rails environment is loaded, and loading that environment means initializing the plugins that require the gems …

Consider this example, assuming a Rails application that uses the “uses_foo” plugin:

in config/environment.rb:

In vendor/plugins/uses_foo/init.rb:

And in the local gem repository, you have installed both

Because plugin init takes place before Rails initializer.run is invoked, it will load foo_gem-1.1, since the plugin does not specify a version of the gem to load. (Rubygems always loads the latest version in that case.)

But, since Rails believes (via config/environment.rb) that your application must have foo_gem version 1.0, the initializer will fail, and the app will not start. And all manner of things that depend on the Rails app environment (like rake tasks) will fail as well.

From what I can see, there are several possible ways to solve this problem:

1. The Rails code to handle/manage gems could be unpacked into the Rails application when the application was created by the Rails command. This means that code can’t rely on any other parts of the Rails framework. Considering that the Rails core team are all staunch advocates of DRY, this isn’t likely.
2. The plugin could mange its own gem requirements, and provide a mechanism to install the gems it needs. In fact, it should be possible for a plugin to install the gem into vendor/gems via install.rb. I’m not sure why this approach hasn’t been taken; if there are technical reasons for not doing it, I’d like to know.
3. You can, of course, simply require an out-of-band process to handle gems. This is ugly, but I suspect it’s what a lot of us are doing these days.
4. Finally, you can just give up on plugins. That seems to be the way the Rails world is going; plugins are being abandoned in favor of gems. (I’ve seen several gems, including haml, that actually turn the mechanism on its head, and provide a tool to inject a plugin into your Rails app once you’ve installed the gem.

Once you have figured out how to get the required gems for your plugin installed, if you decide to use the config.gem tooling in Rails, be sure to use version range specifiers rather than specific versions. In fact, plugin authors could probably save us all some grief if they used the :version modifier on the require directive for the gems they depend on.

Tags: Rails, Ruby, plugins, gems, problems

Why software fails

The rise of complex software systems sometimes creates the impression that software failures are inherently complex, but this is not the case. All software has bugs, and the reasons why software fails are surprisingly straightforward. Like the plot in of a mystery novel, there are many variations but only a few theme.

1. Software fails because requirements are missed. This category of failure has two subcategories.
1. We didn’t understand enough. Communications sometimes fail. Requirements are missed when they are communicated improperly.
2. We didn’t know enough. Communications break down. Requirements are missed when they are not communicated at all. 
2. Software fails because it is implemented incorrectly. In other words, We didn’t test enough. (Sometimes there’s a confluence between failed implementation and communication problems.)
3. Software fails because the assumptions we’ve made about the environment in which it runs prove incorrect. Or, to put it another way, we didn’t plan enough.

Reflecting on Agile Principles

Agile development is the latest rage in the software industry. The last two organizations I’ve worked for have claimed to be implementing the Agile method in their development projects. Here’s a review of the principles behind Agile, with some comments and observations “from the field” based on my experience. This is not intended as a critique, it’s just my personal reflection on Agile as a developer “in the trenches.”

Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.

As a software development organization, our highest priority is customer retention. That not only means me must deliver valuable software early and often, but it also means that the code we deliver must be maintainable.

Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage.

We do welcome requirements changes, because they represent new business for us, but we need sales/management support for this. We need management that understands that while we will respond quickly and accurately to changes in requirements, those changes will sometimes impact the delivery schedule.

Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.

This is a valuable goal and a dangerous one. Done correctly, frequent delivery gives the customer a visible demonstration of the progress of our work, and gives us both the satisfaction from positive feedback and the corrective from constructive criticism. There is a great danger, however, that we will become overly pragmatic, and ship code that is byzantine in complexity, inadequately covered in tests, or even that we don’t fully understand, simply because it does the “right” thing on the UI side when we click the buttons during the demo.

Business people and developers must work together daily throughout the project.

This is important. It is also impossible unless two things are true of the relationship between the business people and the developers:

• There must be trust in both directions. Business people must trust that when developers say something should or should not be done (like having standards-compliant markup, or not shipping the prototype as v1.0), they are doing the right thing for the business. Developers must understand that when their “pet feature” is left on the backlog because the client’s requirements have shifted, they need to adjust their work patterns.
• Along with this, there needs to be a clear deliniation of responsibility. “It will take 3 weeks to code with current resources and staff” is a statement that only developers can make. “We are reallocating staff away from your project” is a business decision.

Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.

Trust is essential to a successful software development effort. The only thing wrong with this statement is that “trust them to get the job done” is not tagged as a level-one header, boldface and underlined.

The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.

This is the weakest principle, from my perspective. There are times and situations where face-to-face is exactly what you don’t want. High-speed internet connectivity and modern development tools can make telework possible, even in pairing environment. Communications mechanisms (chat rooms, IM, email, wiki) that provide audit logs are often better to face-to-face. Also, it seems to me that the ‘opportunity cost’ of interruptions for face-to-face communication need to be considered.

Working software is the primary measure of progress.

This is overly vague. “Working software” needs to be defined. If we allow that requirements are fluid, then we cannot define “working” as “meeting requirements;” At all costs, we need to avoid an overly simplistic definition that locks us into “the UI performs as expected” as the sole measure of “working.” If I was going to rewrite this, I would put it as “Maintainable working software with adequate automated tests for all use cases is the primary measure of progress.”

Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.

In reality, people don’t work this way. The reason that most estimates of programmer productivity measured in “lines of code per day” are so low reflects this. Nobody sits down and puts out 2.8 lines of code every day of the project. In reality, the chart of effective code produced has spikes and valleys.

Continuous attention to technical excellence and good design enhances agility.

This doesn’t go far enough. It should have appeared earlier in the list. Good design is essential to having a working, maintainable product. Continuous attention to excellence is what distinguishes good developers from average ones.

Simplicity–the art of maximizing the amount of work not done–is essential.

Again, this should have appeared earlier in the list. Excessive complexity is a noose around a project’s neck.  I would state this much stronger: Avoid unnecessary complexity. Implement the simplest possible code
to meet the functional requirements first, then optimize your method.

The best architectures, requirements, and designs emerge from self-organizing teams.

This doesn’t go far enough. It’s not enough to let teams organize themselves. Teams need to be made up of people who:

• have the communication skills necessary to understand requirements when they work with the customer, and
• understand the value of good design,
• and are willing to go back and redesign when a previous design approach proves inadequate or flawed.
• are willing to take the time to understand the architecture, do the work of systems architecture, or
• at least delegate a member to that task and listen to them.

At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.

I agree that this is a worthy objective. In practice, I find formal reflection meetings to be less than productive; they then to degenerate either into blame focus exercises, or become “SWAT” team operations to address a single issue.

Tags: software, development, process, agile

We have a 2-11 in progress.

Michael Dubruiel, husband of well-known Catholic blogger Amy Welborn, passed away unexpectedly yesterday. He was 50. Prayers for Amy and her children today.

Next from the Sierra Club, instructions on where you can drink which beers.

Still, some environmentalists remain unconvinced. New Belgium now distributes its beer in 18 states — a point not lost on Will Walters of the Rocky Mountain chapter of the Sierra Club, who would prefer to see companies working more locally.

“I have seen Fat Tire in far flung places in other states where it shouldn’t be,” Mr. Walters said.

Brewer Learns Lesson About Green Marketing - Green Inc. Blog - NYTimes.com

I’m all for sustainable environmental practices, and I like the idea of slow food, but still somehow it bothers me that greens feel obligated to chastise businesses that are at least trying to be sustainable with criticism that’s misdirected. Apparently, trucking beer from Colorado out to Ohio or New York is a bad thing.

For what it’s worth, craft brews at my local store seem to all be priced about $1-2 more than the mass-produced major brands, and the price doesn’t seem to reflect the transportation distance. Of course, the nearest local microbrewery has prices are reflective of the region. (For those of you who think they are way over-priced, for comparison I offer that a round of mixed drinks for four people in NYC recently cost over $60, and that doesn’t include a tip for the bartender.)

Tags: beer, environment, green

Google bows to pressure, removes “harmful” label from the web.

For one hour, the truth was out there.

On Saturday, Google quietly acknowledged what we all have known for years: the internet is bad for you and your computer. Sadly, Google soon caved in to pressure and removed the labels, claiming they were the result of a “human error.” » Google’s flub: Do we have a Web monoculture too? | Between the Lines | ZDNet.com.

But we know better, and we should admit it. The web is bad for you. Stop using it.