Assembla Blog: Category Agile Development

How to select good trial tasks

andy@assembla.com — Sat, 27 Jan 2007 11:53:00 +0000

We use trial projects to help find the best developers to work with. We hire candidates to do paid trial tasks from our live projects. In working with the candidates, we understand how good they are and how well the they work in our process. We try to sign the good ones for longer term contracts. Itâ€™s a lot of work, but it is well worth the effort.

Sometimes, clients want to do this themselves. Or, they want us to perform in a trial, which is a hassle for us, but a good idea for them. That brings us to the subject of how to select trial tasks. To make the process work, we need good trial tasks.

I find that a managerâ€™s first instinct is to create a â€œtestâ€? task. Itâ€™s a small task that can be assigned to lots of candidates. That way, they save time by doing one specification, and they get the advantage of being able to compare results from multiple candidates. Or, they select a task that they have already specified and completed. That way, they save even more time because they donâ€™t have to do a new specification, and they can compare the candidate result with the previous result.

I donâ€™t use either of these approaches (a test task, or an already completed task), because they make the task meaningless. This causes two problems.

Good developers (including us) are busy, and work very hard, and donâ€™t have time to do meaningless work.
These trials fail. As a busy manager, you will never bring yourself to pay enough attention to a meaningless task to get a good result.

So, I only assign trial tasks and features that I intend to deploy, or that are R&D / prototyping for something that I intend to deploy. I only assign meaningful tasks. I think to do otherwise shows disrespect to the hard-working candidates I hope to work with.

For trials, I take a real system that I am working on, and ask for a meaningful improvement. Essentially, I assign tickets out of our live projects.

I hear three objections to this approach

â€œI need the deliverable. I canâ€™t risk it with a new guy.â€? If that is true, then hire two guys to do it.

â€œItâ€™s too much work to specifyâ€?. Thatâ€™s true if you are working with mediocre developers. However, if you are working with people who require precise specifications, and they just do coding, they arenâ€™t saving you much time. They would save you a LOT more time if they could take the general idea and a do something with it. So, it makes sense to test for that.

â€œMy systems are too complex for a quick trial.â€? Thatâ€™s true if you are working with mediocre developers. If you are working with good developers, you should be able to throw them into a complex system and get some forward motion pretty quickly. From a qualification point of view, complexity can help you. If the build process is such that you canâ€™t get new people set up easily, then you will benefit from improving the build process. That in itself can be a trial task.

Build an online game in 7 days?

andy@assembla.com — Sat, 27 Jan 2007 11:34:29 +0000

This article on Gamasutra provides a variety of “juicy” hints and examples from “4 Grad Students Who Made Over 50 Games in 1 Semester.”|

My last article on Hyper-agile product development focused on managing a complex project for minimum time to market – a sort of top-down approach for managers and teams. This article is a great follow-on for the individuals involved, a bottom-up approach to prototyping and testing individual features.

Lessons for Hyper-Agile product development

andy@assembla.com — Sun, 07 Jan 2007 21:15:00 +0000

Recently we were called upon to build and launch a new Web product with a hard deadline in a seasonal business. By the time we started our work, only three months remained until the planned launch date. It was a big system. Our mission was to build a new and greatly enhanced version of a product which had taken 15 months to build in its last iteration. So, we had a benchmark.

In this case, we ran more than four times faster than a similar project implemented last year with older tools and development methods. We also came in on-time and under budget, and we ended up implementing features faster than we could figure out how to use them.

How did this happen? Want to try this at home? I will share our lessons below. Good luck.

Donâ€™t do things that take time to arrange
Donâ€™t get dragged down by old code
Establish the data schema
Pile on with new team members
Use a ticket list / work queue
Build something now, even though we know we will have to rebuild
Donâ€™t split the codebase into components
Daily releases instead of stabilization periods
Make a usage dashboard
Web services are your friend
Hire users as product managers
Donâ€™t create obstacles: Users provide the data
Documentation on a wiki
Find the one thing people will use

Here is the expanded version, for people who actually do this for a living.

Donâ€™t do things that take time to arrange

We get big time savings by not moving people around. Travel takes time to arrange, and it takes time to do. People should be working, not moving, and they should start now. I donâ€™t do interviews. I found that even arranging a phone interview takes five to seven days â€“ a big chunk of a 70 day project. Instead, I let people start work and I evaluate their work later. I also donâ€™t set up conference calls for the product team. In my observation, time spent on the phone is a good indicator of communication problems, and is best avoided by improved management and communication.

Donâ€™t get dragged down by old code.

I believe in doing a lot of research and trying to adapt existing open source code, as an alternative to diving into new development. We spent a week of our precious schedule taking a in-depth look at open source alternatives. We considered a system with a friendly academic license and $4M worth of development behind it. What we found was big, complicated enterprise software with lots of heavy baggage. We talked to people who were adding to Sakai, and found that they were typically taking 8 months to do things that I thought should take a month. We discarded the existing product for the same reason. We moved on to a more agile architecture and never looked back. I had to change my thinking because, in the new world, it often is faster to build from scratch.

Rails is simpler than java for this type of application, and the build was much faster as a result.

Establish the data schema

It is easy to add to a data schema, but difficult to change it. I sometimes visit old clients and find that the people, products, and code that I knew are long forgotten, but the data schema remains. Data schema is the one piece of up-front design that we do before diving into the build.

Pile on with new team members

It is possible to bring in a lot of new team members and let people pick their own tasks. Using this technique, we were able to triple our output in three weeks. This approach is the exact opposite of the “Mythical Man Month” recommendation, which holds that adding new team members imposes a cost of learning time and management time which will delay a project that is on a short time schedule. In my experience, adding team members DOES delay a project, if you need to guide them and manage them. The pile-on method works if the new developers are good and can find their own way. We are helped by our â€œinspired by open sourceâ€? methods, which were developed to serve large teams with loose central management, and the fact that some of the new developers are very good. We can weed out the rest a few weeks later.

An obvious related lesson is to work with good people. I am not even going to put it in the list of lessons from this project because it is far from a new lesson. A good developer seems to be about 5 times as productive as an average developer. In this project, there were several cases where a team member spent 2 weeks working on something, did not finish it, and then found it replaced by something our best guy finished in 2 days. Given the potential gains, most project managers donâ€™t invest enough in recruiting and retention.

Use a ticket list / work queue

We put all of our tasks, bugs, and new requests into a list of “tickets”, often called a work queue. We use Trac. In fact, we use Assembla.com. We sort the tickets by priority and milestone, so the tasks from the next milestone are on top of the list (queue). Itâ€™s easy to figure out what to do. Just look at the list of tickets and pick something near the top. You can try to adopt a more rigorous agile methodology, but you will find that it is hard to improve on this simple and effective technique.

Build something now, even though we know we will have to rebuild

We started building copies of the old product, and copies of features that we saw in other products, before we even had a coherent visual design or plan for the new product. Then we ripped out a lot of the guts and rebuilt it. This was still faster than waiting around for a coherent design and product plan. Why? In this new environment, development is faster than product management. From a product management point of view, it is faster to ask for changes than to figure out what the product should do from scratch. It takes forever to build something that is unknown, but if you know what you want, builds are fast. So, the two steps (copy and redesign/refactor), with clear instructions for each, are faster than one build from a blank sheet.

Donâ€™t split the codebase into components

In this application we needed several types of servers that were different â€“a permissioned application, a public portal, and a central user registration server. We lumped them together in one repository and one application with configurable â€œrolesâ€? that turn features on or off. This made the code a bit more complicated. However, it made other aspects of managing the app â€“ setting up developers, testing, staging, deployment â€“ a lot simpler.

Daily releases instead of stabilization periods

In a normal Web 2.0 style release, you just run beta releases of increasing quality, until you are ready to do a bigger announcement. In this case, I didnâ€™t think we had time for such gradual stabilization. I initially recommended that we do a code freeze with a week of testing and stabilization. The resulting product might not be perfect, but at least it wouldnâ€™t crash. We started with a daily release process and no code freeze, and it worked well enough that we didnâ€™t need to a code freeze and stabilization period. Itâ€™s not bug free, but we fix the bugs as fast as people notice them. I now have abundant evidence, from several projects, one of them a big enterprise ASP, that users will tolerate a product which comes directly from developers, every day, with minimal testing, if the time to release a bug fix is less than 24 hours.

Make a usage dashboard

Program a custom dashboard that shows registration and usage statistics. Everyone on the team will use it every day as soon as the beta period starts.

Web services are your friend

Arranging integration with external systems can absorb a lot of calendar time. We were able to use SOAP services (easily offered or called in Rails) to make it go faster than I expected.

Hire users as product managers

Product management â€“ figuring out what the product should do â€“ became the bottleneck early on and remains the bottleneck. Using the above techniques, the developers pull requests off the ticket stack and implement them as fast as we can make the requests. Our strategy to deal with this is to hire users as part-time product managers, and let them each work with a mockup team to redesign the visual parts of the app. With the distributed team system, itâ€™s easy to integrate part-timers like this.

Donâ€™t create obstacles: Users provide the data

We took out the centrally controlled categories and allowing users to just type their own tags for the content that they entered and shared. I think we would still be waiting if we had to build our own categories. We got a surprise when we released with a comprehensive list of US data, and we had immediate requests from dozens of countries. I now think that users should be able to post additions to any part of the database if they need to. Why block them? Sometimes the additions can be released directly, and sometimes they may need to be approved by an editor.

Documentation on a wiki

If your application changes every day, you need to put the documentation on a wiki. If you set up the documentation wiki early in the development process, you can start linking to it from inside the app, just like embedded help pages.

Find the one thing people will use

If you can find the one feature that people will use and adopt, it makes the whole system simpler. So, release early and often and watch your dashboard. Find that thing.

To meet an impossible deadline, cut your time in half

andy@assembla.com — Sun, 10 Dec 2006 16:53:00 +0000

A major consumer goods company told me last Tuesday that they wanted to put in a gift purchasing system. This involved a couple of roles (giver and receiver), a credit card gateway, Web services for fulfillment, email reminders, etc – a serious application. I would normally want about 6 weeks to build and test such a system. They told me that wanted it for the following Monday – 6 days away – and they were willing to pay a premium for that. I took the job, and I told everyone that it wouldn’t make sense to do it in six days. Instead, we needed to do it in three days.

When I plan a software development job that has a deadline, I use a 50% scheduling rule. We need to have a relatively complete, working version of the software available for testing when we are halfway to a full release. If I want to do a product release 3 months from now, I plan to have a testable version of the product in 1.5 months. The halfway release can be rough and even ugly, but it needs to do most of the things the complete release will do. This rule ensures that the incremental release process can work its magic. You have half of the total time available for finding and fixing problems and making improvements.

What if someone gives you a seemingly impossible schedule? What if you don’t have enough time to get even the first version out? In this case, I still use the 50% rule. You force yourself to deliver something in half the time.

This just recognizes reality. In reality, no product is done in its first testable release. If you think you can do a release without spending half your time on incremental improvement, you are probably wrong. If instead you make a simpler product and give yourself a few cycles to make it work properly, you will look pretty good. As an added bonus, you motivate the business guys that you are working with to deliver all the stuff you need.

We posted our midpoint release early on Friday. It didn’t look great, but it had all of the transactions and it worked. I got my gift. Thursday was an all-nighter, but now it’s Sunday, and I have time to write this blog post.

Roadmapping - how your product finds its way

andy@assembla.com — Thu, 28 Sep 2006 21:43:00 +0000

A startup will often live or die based on its first product release. Did it get released? Did people find it useful? Good roadmapping dramatically improves your chance of getting to â€œYes!â€? on these critical questions.

Read about my roadmapping technique over at th On Startups blog.

I think it’s my most immediately useful post so far. It’s a technique that has helped me work with individual entrepreneurs, my own products, venture funded companies, and giant corporations. I gave it to Dharmesh at On Startups because he has a much bigger readership that could benefit from this technique.

So, check it out.

Two kinds of teams - shared leader, and single leader

andy@assembla.com — Sat, 23 Sep 2006 18:29:00 +0000

Should team task assignments be controlled by a project manager, or should each team member create and accept tasks? You can get a big increase in peformance if you know when to use each of these methods of organization.

A client wrote: “my plan was to have them work separately first and then after they passed the one week test to have the ones we like work together. Is this not possible? I think we have a fundamental difference on the idea of a “team.” Your model is a self-organizing group of people who work on the same code. My model is a group of people who work towards the same goal by writing individual components while making sure all the components interface correctly.”

I wrote back:

You might be interested in this: The Discipline of Teams.

A friend of mine, Doug Smith, wrote a series of books starting with “The Wisdom of Teams” and “The Discipline of Teams” in which he looked at how teams are managed. He believes that there are two different management structures. One is the “team”, which is a group of people that self organizes in the sense that they pick their own tasks, and any one of them can lead the effort if needed. A team can form if people have a strong sense of a shared goal. The other structure is the “single leader group”, in which a group leader figures out who should do what task at what time. That’s a classic management structure that can work even if the shared goal isn’t clear to everyone. According to Doug, there are specific circumstances when you would choose a “single leader group” versus a “team”. I can see you making this type of decision at Grazr. Your work with Kurt is part of a team. Your work with the new guys is single-leader.

I liked these ideas, learned from them, and invited Doug to be on the board of directors at PowerSteering Software, where we hoped to implement tools for both methodologies. I’m a big supporter of the single-leader methodology when a leader is in place to make it work, and I use it myself when working with new guys.

Jon Udell Podcast: - A conversation with Andy Singleton about building global teams

andy@assembla.com — Sat, 20 May 2006 22:53:00 +0000

Jon Udell included me in his Friday podcast series this week with a conversation with Andy Singleton about building global teams . Jon has been putting up with me since 1993, when he edited some freelance articles that I wrote for Byte magazine. In 2003 he coined the term “dynamic development” to describe the work I was doing. He has recently been commenting on user innovation and the power of human networks to do work beyond open source sofware. In this conversation, he turns me on to Yochai Benkler’s The Wealth of Networks, and he cautions against a “walled garden”. Maybe we’ll be able to work together on a universally portable user profile.

Ancient Greek Craftsmanship for Software Teams

andy@assembla.com — Sat, 20 May 2006 21:16:00 +0000

I have been reading a book called The Greek Commonwealth â€“ Politics and Economics in Fifth Century Athens. Originally published in 1911, itâ€™s not out of date. According to the author, â€œit is natural for human beings to enjoy using their own best faculties. Men never felt that enjoyment so keenly, or put so much high effort into its attainment, as in the workshops of ancient Greece. If you seek a proof, go look through the shelves of our Greek museums. There is hardly an object that they made, however rude, but bears on it, sometimes faintly, sometimes with speaking clearness, the touch of the spirit of Art.â€? How can we bring that spirit to our work, every day?

This author (Alfred Zimmerman) also writes: â€œOur modern industrial system, with an ingenuity so wicked that one might believe it to be deliberate, has contrived to take the joy out of craftsmanship, and so to choke up the very springs of art. It has replaced, wherever possible, the delicate skill of the human hand by inhuman machinery, and the independent thought of the human brain by soulless organization. It has removed the maker or producer from all association with the public for whom he works, and substituted a deadening cash-nexus for the old personal relationship or sense of effort in a corporate cause. Above all, it has taken from him his liberty, and forced him to work for a master who is no artist.â€?

Alfred Zimmerman, a long-dead Oxford don, could lead a workshop on how to build modern, high-performing software teams and startups. Here are some of his observations about the craftsmen of ancient Greece:

When the Greek city states hired craftsmen, the foreman and the well-know artists received the same rate of pay as all of the other men on the crew, including the slaves. This rate was set for a long time at one drachma per day. There were no social barriers at work.

The most talented competed not for money, but for honors, which in Athens included a golden crown.

Competing craftsman clustered together, sharing their clientele, their stories, and their knowledge.

They had enough leisure time for stories, and aspired also to be citizens and soldiers.

Greek workshops tended to be small â€“ a craftsman and several apprentices â€“ and they were never bigger than 20 people. The master was always an artist, not a manager.

Like many before me, I believe that the pre-Christian philosophy of the ancient Greeks gives us vision of what a man can achieve on Earth, as a citizen and father. I draw inspiration from Odyssyus, the ultimate survivor. I am often amused by the goofy anecdotes of Herodotus. Sometimes I read the endless political debates of the democrats, demogogues, and tyrants, learning how great nations can be hijacked by small-minded men, and how the small-minded can eventually be cast off. That sounds relevant to my reading of the daily news.

What American cannot be inspired by the story of the war against Persia? The Greek citizens came out of their stone huts and small towns, and stood against the army of the greatest and wealthiest imperial power on the face of the earth. There was the last stand of the 300 Spartans, the desperate evacuation of Athens, the high-stakes maneuvering at Salamis. Then came victory, the Parthenon, and science as we know it. 2000 years later the minutemen had this very story on their minds as redcoats marched into Concord.
Emerson gives us the scene:

By the rude bridge that arched the flood,
Their flag to April’s breeze unfurled,
Here once the embattled farmers stood,
And fired the shot heard round the world.

Introducing Assembla.com and Breakout

andy@assembla.com — Sun, 30 Apr 2006 10:21:00 +0000

Over the past few months, we have done a lot of work on Assembla.com, our online service for building and running distributed software teams, and Breakout, the underlying platform for professional networks. Now, we are ready to share. If you have suggestions for user groups that can benefit from this resource, or bloggers that might be interested in it, let me know at andy@assembla.com Read on for the story->

Assembla – for dynamic, distributed software teams

Assembla.com provides workspaces and services for distributed software teams. Techcrunch readers can think of it as a sort of construction kit for Web 2.0 development teams. These teams are usually distributed, and often composed on the fly for a particular project. We call this approach inspired by open source.

I have been working on this approach for years, doing rapid development with great developers and low fixed cost. Now, we are bottling the magic and making it available on a larger scale.

If you are managing software projects, you can create a project workspace on Assembla.com to manage your code, documentation, and team permissions. This creates a secure IP asset that you build a team around as needed, with no fixed costs and unlimited team scalability. You can also advertise opportunities publicly or selectively. If you work on these projects, you can create a personal space, post a profile, join teams, and engage in public and private discussion with potential employers.

The online service is a work-in-progress with many rough edges, but it is feature rich in comparison with other collaboration services. Each workspace offers a wiki, discussion, issue management, feed aggregation, email alerts to the team, team invitation and permissioning, file store, Typo blog. Those are generic features for any professional network. For software teams, we have integrated Trac and Subversion, with instant activation and single-sign-on permissions. There is currently no restriction on the number of spaces, files or pages allocated to a particular user.

The service is free. We make our living supplying premium applications, and full-service distributed team recruiting, management, and development.

A Platform for Professional Networks

We describe the underlying Breakout platform as Social Software to Get Things Done. Our approach is loosely like cramming MySpaces together with real enterprise software. Over the past few years we have learned that certain kinds of software – social software for exhibiting profiles and starting discussions – is very easy to adopt. We get people engaged with easy-to adopt profiling and collaboration features, and then we let them progressively crank down on permissions and ramp up application capabilities to make real working teams.

We describe the result as a “professional network” – a population of people who work together in a variety of permission groups, ranging from global (eg an open source project), to enterprise, to small project teams. We provide mechanisms to engage these professionals, help them organize themselves into groups and projects, and support them with on-demand applications, which can be added to the workspace with integrated permissions in the same way that Trac and Subversion are integrated with assembla.com.

Our first customer for this platform is working on a network to manage the response to biological threats such as an avian flu epidemic. So, we believe social software can help get very important things done.

The software is available for enhancement and customization under an open source license.

Team

Assembla’s experienced management team came together in order to bring “inspired by open source” practices to enterprise software. Andy Singleton was the founder and CEO of Cambridge Interactive and PowerSteering Software, now the leading application for corporate six sigma deployments. Sesha Pratap was a founder of Centerline Software, which he grew to $20M as CE. Jon Stillman has 15 years of experience leading technology projects at GE, topped of with a stint doing strategy for Divine Interventures and a venture partner position at High Peaks Ventures.

When hiring - try before you buy

andy@assembla.com — Sun, 30 Apr 2006 09:41:00 +0000

To me, it is self-evident that you should test any co-worker, in the job that they will be working at, before you make a permanent offer. I was surprised to see how much criticism Dharmesh Shah picked up for recommending a two month probationary period for new hires, during which the employer or employee could withdraw without negative consequences. Much of the criticism was along the lines of “You should do better interviews”. I suspect those critics have not hired many people themselves. I have been doing it for 20 years, and my hit rate for picking really good workers (who don’t bail out) from even the most extensive interview process is still only about 50%. By persisting in finding ways to run trials, I can move that to 90%. This makes an enormous difference in startup risk and ramp time.

A 50% failure rate sounds high, but if you actually add up the reasons that people leave a job within the first three months, it might even be an underestimate. Reasons include:

They aren’t very good in the role and you encourage them to leave, or don’t renew an agreement.
You can’t work with them or you don’t agree with each other.
You change your priorities or you can’t afford them.
They determine this job isn’t a good opportunity, because the company or role has limited prospects.
They determine on their own that they can’t do a good job, and bail out. Or, perhaps the advancement path isn’t clear, or the job role isn’t the best one for them. People are surprisingly efficient at figuring out their chance of success in a job.
They find they can’t commit the time you are asking for.

There is one place that I disagree with Dharmesh. He’s recommending a two month trial period. I think that is too long.

There is a some evidence, discussed in my article Blink and the art of hiring the best, that people form strong opinions within the first two minutes. However, those are inaccurate opionions that are not based on actual work in the chosen role. Those are the inaccurate opinions that come out of the interview process.

You have to give people space to work. However, once your mutual opinion forms, it is unlikely to change. So, once your opinion forms, you should act on it. This will not take anywhere near two months.

You will form an opinion of someones effectiveness, and they will form an opinion of you and their chances of success on the job, within the first week of work. If the person is working remotely, it may take two weeks to stabilize the opinion.

My goal is to put someone in a working role BEFORE I interview them. That way, I get the acccurate information, without the inaccurate information that comes out of interviews. This is possible with developers, but harder for other positions.

If you do hire someone and they quit or are released after a two month trial, you just did something very expensive and very risky. You spent from $10K to $80K on recruiting, salary, admin time, severance, etc., and lost two months. It’s a huge risk that few startups can afford, and it’s avoidable.

I find that getting good at doing the one or two week trials makes an enormous difference in startup ramp time. This can move the expected time to fill a position from two months to two weeks.