main mozilla tech

Thoughts on hybrid projects

One of the biggest challenges of all open projects is how to effectively manage the most precious and most crucial resource they have – people. We approach this by building a project cultures that are open and participative,  tools that allow us to operate in this open environment and we make whatever possible to make it easy for new people to learn how to blend in to work with others in the way that matches the culture of the project.

In fact, open project became extremely efficient with managing their projects with all the new tools, like trac, mantis, bugzilla, revision control systems, wikis, forums, blogs, planets, newsgroups. You can manage privileges, discuss in groups, build subgroups, work in a cloud, share code, brainstorm, and do this pretty efficiently. The only trick is that it works well on a smaller scale.

The challenge becomes trickier when a project achieves what we tend to call a “success” and in result one of the two happens:

  • The project is growing and it becomes impossible for anyone to know everyone around and maintain a track of all events in the project (our brains are not suited to keep the track of more than 20-30 people around us)

It’s a challenge because it becomes inevitable that each member will loose track of the project “overall” dynamics. People differ in a degree to which they want to maintain a control or knowledge about their project  as whole but each of us would prefer to understand more and know better on what is happening around, especially if it may interfere with them later. It gives people a sense of predictability. They feel safer. Operating in a big environment reduces ability to understand and predict what will happen and in open, decentralized environment the decision making process is distributed too so no matter where you are in a structure, if you are a module owner, peer, l10n team leader, bug hunter, evangelist, employee or not, CEO or QA things are happening all around you outside of your scope and may influence your work and may hit you later.

Overall, the growth of any project, while usually a highly expected result of a hard work and success raises new challenges and raises unpredictability inside the project (which may be good or bad, depending on the kind of people you have in your project), but also weakens social control which makes it harder to maintain same level of sociometric density.

  • The project is gaining a formal central place like a foundation in the middle or a formal board

In result of the growth one of the solutions to many challenges, including problems with management, and project focus is to found a formal entity that take a maintainance role over project either by positioning itself in the center of the project or on the side (by taking care of a specific segment of the project scope – like Canonical in case of Ubuntu).

While such move is usually an extremely good thing raising efficiency and sustainability of the project and brings ability to fulfill project goals to the new level  (not to mention inclusion of many more formal elements like documentation, brand management, quality management, financial management etc.) it also introduces new level of complexity to project dynamics.

The reason for the latter one is that we’re gaining another dimension of roles in the project. Traditional roles – leader, owner, peer, energizer, hacker, bug hunter, tester, community animator, evangelist, spokeperson, observer, PR maintainer, information distributor, accessibility expert, catalyst, coordinator, harmonizer, gatekeeper, compromiser etc. keep one demension – community roles, while on the other dimension we suddenly have such roles as a contractor, employee, manager, Board member, CEO, VP, intern etc.

While those two may easily blend if we not forget which one is attached to which dimension (ideally being a manager should mean nothing for the project by itself, should only mean for the formal structure and in the ideal world, to keep layer synchronization, should overlap with one or several of the group roles like a leader, animator, owner, information distributor etc.), they are shifting the dynamics in the project and require a conscious awareness to reduce an influence on how community/project role distribution happens.

The project which achieves this stage, and maintains those two layers of roles (or even more than two – think of multiple community circles and sub-projects inside a meta-project where people are keeping their roles) is called hybrid. A hybrid project is a project that maintains several layers of communities and in particular the two mentioned above – organization structure, and project structure. A hybrid project has a unique opportunities and challenges, and has to put a lot of effort all the time not to loose its balance, not to kill or destabilize one of those in favor of another (in particular it is easier for organization structure to destabilize project structure).

Two of the most crucially influenced project patterns by shift from open project into a hybrid model are distribution of information and distribution of influence (often referred in sociology as distribution of power, but I don’t like the name in this context).

Information distribution

Let’s talk about information distribution in relation to open/hybrid projects.

Information is an important tool and precious resource in every project, no matter of scale, goals or project type. Human beings, in particular, are facing an exciting challenge with it, because evolutionarily information was an important factor for the survival chance – knowing where is water, where is enemy or where is a partner drastically raised chances of success. Today, we’re becoming surrounded by information but this change from underinformation to overinformation is happening too fast for our brains to cool down temptation to get more and more information whenever possible. We’re addicted of information and if you ask yourself why you’re reading this blog post, you’ll get to what I mean. You read it because you’re curious. Curiosity is a term we use to describe the desire to know more, always get more information.

Because, as we said, our brains are not ready to process all this data that we can find every day on the Internet, we’re suffering from two effects instead of one. First, there may be an important piece of information for us that we do not have available, second there may be too much information around us and we need to select which ones are valuable to us. Such selection process is time and resource consuming and any mistake may be very dangerous.

However far it sounds from our everyday life, especially in a volunteer open project like Mozilla, it is a sole reason why we track other people bugs, read their blogs, subscribe to planets, feeds and other tools that help us “stay up to date” – reducing risk of missing an important information and trying to raise predictability around us.

In the early days of each project, when the scale of the project is limited, we can easily keep coverage of the whole project just by simple p2p interactions and avoiding longer desynchronizations from the project central piece (be it forum, wiki, mailing list, or pub meetings). With the project growing in people, scale and resources, its becoming harder and harder to maintain the same level of predictability and we’re loosing the luxury of perceived control over the project.

If we additionally avoid the temptation to drift into a closed, top-down model of control where someone up there is maintaining full proximity of control and is a master of life and death, and instead drift into open, decentralized model with no top-down control per se, where any kind of control is drastically limited we are becoming desperately dependent on tools that give us needed information and filter the information that are invaluable so that we can process what’s valuable and meaningful to us. Only if we succeed here we can achieve a state of mind that we call “feeling the project” – understanding its dynamics.

So we keep dancing between streams of information trying to select whats important and make sure that we avoid filtering out, or for other reason not getting those important for us. If we fail, the feeling of lack of personal security and understanding grows and this may lead into lack of emotional engagement in the project and loosing satisfaction from being part of it.

In a hybrid model, the information is drifting into being distributed unequally. People can physically be “nearer” or “further away” from information sources, and people higher in the organization structure layer may gain easier access to information that those high in the project structure. It happens naturally, without any conscious effort from anyone and requires effort to be minimized. Anyone who’ll spend a month in foundation office will gain unique knowledge about the project they are part of that would be much harder to gain by someone external to the foundation but high in the project role ladder.

Of course in decentralized model its not black/white like in a closed model. A lot can happen outside of the HQ walls, and people inside still have to keep dancing to get their level of comfort, but it is visibly easier and cost less to know more while near to the center. In result we can say that organization layer structure is negatively correlated with cost of gaining access to information and ability to select important information from the stream. This is true even for closed organizations where remote employees are suffering from the very same syndrome which destabilize it even without multi-layer open models.

How can we address that? There are two major methods which I see in most hybrid projects.

First, people tend to work to overlap structures. When possible, community organizer should get employed to work on communities, code module owner is becoming manager of the module etc. On the other side people employed from outside are working hard to gain status in the project structure and find their place not far away from their formal role. Additionally, head figures in project and formal structures are working hard to maintain clear separation between the layers and ensure balance – for example new MoCo employee by default has no greater access to Bugzilla than new volunteer. They both have to work to gain privileges or get Hg repository write access.

Second, tools. We build plenty of tools that are supposed to support this effort, by giving equal privileges, no matter where physically the person is, and ability to give access despite formal status (open meetings, calls, video streams from meetings, real life events, IRC channels, bugzilla, open wiki etc.)

Influence distribution

Distribution of influence means an amount of power each person has depending on his/her status in the group. Influence is distributed similarly to rewards. We know many models – we may reward depending on the social status, experience, influence, equally, depending on person needs etc. Similarly we can distribute influence depending on many factors and using different methodologies. In result we should have a rather consistent pattern according to which we may predict amount of power a person has in a group.

In open projects people usually gain power with experience gained in the project (notice: the experience gained before joining the project is not that important here) and amount of work they put into the project. In hybrid projects, its becoming tricky as on the organization layer, the influence is distributed differently than on the project layer. In particular, organization structure drifts in the direction of keeping its own structure and distribution of power that comes with it, and perceives the influence distribution that happens on the other layer as a factor that raises unpredictability and risk. The subtle nature of distribution of power lies in the quote from Benjamin Parker: “With great power comes great responsibility“.

While we well understand that the single most important factor that is required for any open system to grow is an assumption that generally speaking, people are good, we also understand that in order to be sustainable we have to keep sociological conservative approach toward other people. Cold rationale says that even if we are good by vast majority, the minority that is not good may easily cause great troubles and in our approach we have to ensure we do not let those individuals be free to do this harm.

In other words, we can go Wikipedia style of assumption that vandalism is a rare thing, but its better to have solid tools to revert vandalism acts, ban vandals and, as we see with time, grow “safe” versions of articles.

Same goes with power. While we trust each other and we rely on each other without any formal bounds, it is rational to assume that being tied to formal organization raises safety of the person’s responsibility. We put a great effort in hybrid projects to manage it well, and not fall into a trap of delegating responsibility and power basing on this theoretical safety basis (which would, in the end, kill distributed and decentralized nature of the project) but it is a challenge to avoid drifting with the project structure of power into direction of organization distribution of influence where one has more decision power because of his formal role without any correlation to his project status.

An interesting example of a result of a turbulence in such system was a famous story of Firefox 1.0 theme choice. While Ben Goodger was an admired and loved leader of a new wave of Mozilla applications, he has not been granted by the community the privilege to distribute power in the way he did with regard to Firefox theme choice without external community influence. From the project layer, the way the decision was made was clearly a violation of the project structure and appeared like if another dimension of reality took over charge of the project without any warning. Suddenly something happened and was announced to the community which was totally not in line with how the project has been growing. At the same time, the decision making process was totally in line with how the organization structure looked like which caused a lot of discussions and was a valuable lesson (positive long term outcome – hooray!).

Like in the case of information distribution, the challenge here is not to get the project nature be violated by organization structure and it cannot be achieved by direct overlapping of those two.

In hybrid project, central organization serves as a form of government or Task Force and its reason to exist is the trust of the project members. It means that such organization has a rationale as long as the widest circle of project members accepts its leading role in the project. If this would not happen, the project will shift in different direction and the organization will loose its central position or kill the project by trying to maintain it.

But in a healthy project, such central organization is taking central role in the project and is being trusted to maintain the project beyond the level of democratic decisions made by majority of the community. In particular such organization is free to make strategic decisions related to security, financial matters, law, management and employment and preserves the relation with its community by making sure that the decisions are supported and in line with what the community stakeholders wants for the project but is not expected to consult each decision with the community itself – such thing would slow down or even block any progress.

In result by nature, organization structure has more centralized power and its distribution is denser around the center of organization, while on the project layer the power is voluntarily being ceded toward central organization.

Both shifts in distribution require the project to take pro-active steps in order to avoid those shifts harm the project stability. While a lot can be achieved by solid tools that make it easy to maintain the tools serve as central place of the project development, even more fundamental role is to maintain vibrant story inside the organization about what is the project itself and what role the organization has in the project.

When the project is growing, both in size and by forming a central massive organization in the center of its galaxy it automatically raises the risk of the perceived gap between those inside and outside of the centers of action. People tend to assume that because there’s so much going on inside and its all so professional, efficient and sophisticated, they have less and less to say, they know less and less and their potential influence on the project is minimized.

To keep it clear. It is possible for two people contributing to the project in the same way in 2001 to find themselves in very different points on the influence structure and information distribution structure because one of them found a place for himself while the other lost sense on how he can contribute. The only difference between those two people in this artificial example is in their own mind, but the result is very real. One of them “feels” inside, while the other not only “feels” outside but also believes there is no way for him to get inside the project. (by inside I mean into circles of information distribution and influence distribution)

While its clearly false in case of open projects, it is a pure example of self fulfilling prophecy and requires those who feel inside to show those who may feel being “outside” that there is no glass wall between them and all it takes to get inside is to do one step.

I believe that it is impossible to overestimate the importance of day to day psychosocial steps an organization can take to remind and refresh the scope of the project and ensure that both project and organization complement each other, and understands why its vital to proactively distribute more influence and information outside of the central organization toward the far ends of the project universe.

In particular, I believe that Mozilla is extremely successful in this. Either by amazing luck, emotional intelligence or experience from other projects we maintain being inclusive and put effort to strengthen our signals so they can reach far ends of our universe and make sure that anyone interested can get all information about the project without any effort. Things like public access to all weekly meetings of all groups, video streams from standups, wiki notes from meetings, public bugzilla, real life events like Whistler, All-Hands meetings and public irc channel being not a layer between outside/inside but being a centers of the universe for everyone working on Mozilla and being accessible for everyone from anywhere is a huge step in the direction of minimizing the information distribution shift.

Much harder challenge, which Mozilla is also trying to cover is influence distribution. We need to work hard to make sure that people located on the project layer can have their voice being heard and can influence the overall project direction on the similar basis as people on the organization layer in the very similar way, a government has to work on building civil society. We are facing this, and I believe Mozilla is doing exceptional job by opening itself for an external influence and avoiding falling into predictability-exclusiveness trap. We can do even better, I’m sure, but the biggest risk was avoided. Probably somewhere around Firefox 1.0 Mozilla was in the right spot to make wrong decisions, and avoided making them.

But the fact that we avoided making wrong decisions, doesn’t mean that the threats described above are not valid, and we definitely have to remain aware of them every day of our work. All Mozilla circles of communities should feel equally important, and both layers should be aware of their roles and importance.