Click. Connect. Learn.

All posts in Drupal

Community Development and Code Reviews

All of what follows in this post is based on two core principles:

  1. Teaching and explaining a topic is the best way to master it; and
  2. The ideals of a community are best handed down from individual to indivdual as they work toward a common goal.

Toward that end, on May 11th, the Portland Drupal Users Group will be using our monthly meeting to run code reviews.

As a side note, while this writeup uses Drupal as an example, the general principles will work across projects, or in training/educational contexts outside software development.

Peer review

Benefits

Code reviews can help grow the existing pool of talented Drupal developers by providing a documented, more structured way for developers new to Drupal to begin learning the Drupal codebase, and learning how to work within the community that maintains with that codebase. Code reviews can also help alleviate one of the more consistent bottlenecks for developers looking to contribute more to Drupal: the project review process.

Code reviews reinforce good habits. The process of reviewing someone else's code forces a few things to happen:

  • The reviewer hones their developer chops by seeing how other people have solved problems;
  • By reviewing code for security issues, developers get more familiar with the best practices for writing secure code, and security issues in exiting code can be addressed;
  • By reviewing code for proper and appropriate use of Drupal's APIs, developers get more familiar with Drupal's APIs, existing modules get more stable and less likely to break on upgrades, and more people become familiar with how core works;
  • By testing and reviewing code via the Coder module, and/or using Devel as needed, easy fixes get implemented more quickly;
  • Opportunities for documentation and better test coverage can be highlighted as part of the review, leading to more automated review and better documentation.

But, most importantly: the process of providing a review of someone else's code creates a dialogue within the community. Code review is contribution and connection, and these are key elements to working successfully in the Drupal community.

Additionally, code reviews provide a clear starting point for people asking how to begin in Drupal. Not many people are going to start reading api.drupal.org without a clear context or reason to do so. Code reviews provide that context, and the process of a code review provides a structure for people coming up to speed on Drupal development. So, when someone says, "I want to get started in Drupal, but I don't know where to be begin," we now have a simple answer:

Check out this page for some background on giving code reviews, and then review some code. The process of reviewing projects will get you up to speed on how to develop in Drupal.

To summarize, code reviews:

  1. Help new developers learn Drupal development best practices;
  2. Help new developers make contributions and connections within the community;
  3. Help reduce the bottleneck of getting new projects approved. And, it's worth noting that increasing the visibility of code reviews as a valued contribution within the community will likely increase the pool of reviewers into the future.

Getting Local

There are a large number of local user groups. While a lot of work can be done virtually, the value of face to face meetups should not be overlooked or underestimated. Local user groups provide an opportunity for more exerienced and less experienced developers to get together and share ideas. If user groups dedicated one to two meetings a year to code reviews - where more experienced and less experienced developers worked together on code reviews - the habit of code reviews could start to become a recognized norm. Once a developer has become proficient in code reviews, the practice of reviewing someone else's code can just as easily be applied to reviewing their own code.

Over time, code reviews should be as familiar and as recognized as Code Sprints and Documentation Sprints. These are held at every DrupalCon, and are regular occurrences at local Drupal Camps. Given the value of code reviews in developing new talent, increasing the consistency and quality of contrib, improving collaboration between developers, and strengthening the contributions of local user groups, code reviews have a role to play in helping the Drupal community continue to grow, evolve, and mature.

Getting Started

Fortunately, getting started is the easy part. The queue for project applications is open, and you can sort by date to see the oldest projects first.

The Code Review Group is getting up to speed, and you can indicate on your profile that you are actively reviewing code. This wiki page and this handbook page both provide instructions for diving in to a code review.

And, if you are in Portland this May 11, come to the Users Group meeting and get running with some reviews.

Image Credit: "Crystal peer-reviewing Mike's code" taken by Jason Crane, published under an Attribution Non-Commercial Share Alike license.

I'd Like A Server With That Web Site, Thank You: Provisioning Servers and Install Profiles

Over the last few weeks, we have done some work building out an automated installation script that runs on Linode. More detail is provided below, but for those of you looking to move on and read something else, here's the short version: it can provision and tune your server, and install your web site, in about 15-20 minutes.

For those unfamiliar with Linode (see note below), they are a hosting company that provides VPS hosting. Within Linode's hosting environment, people can set up stackscripts to automate/streamline setting up a hosting infrastructure.

Atrium

We worked with Dennison Williams to start the process of building this script. It's also worth noting that, while this script is specific to Linode, the steps within this script can be used as the basis of a script to automate an install in a variety of other hosting environments, from Amazon Web Services to the Ubuntu Cloud.

Some of the steps automated by this script include:

  • Provisioning Linux, Apache, PHP, and MySQL on Ubuntu 10.04;
  • Low-level performance tuning for Apache, PHP, and MySQL;
  • General security hardening, including installing fail2ban
  • Installing Drush;
  • Installing APC;
  • Setting up cron jobs;

And, of course, it completes the process by downloading the codebase for VoiceBox, which can then be installed on the freshly provisioned system.

So, in the space of about 15-20 minutes, you can install a completely configured web site on a tuned, secure, scalable web host. Currently, we have this configured to grab and install VoiceBox, but given that this script can be modified to grab other codebases, it can also be used to automate other installation profiles (Managing News, Open Atrium, Open Public, Open Publish, etc) just as easily. This script makes another aspect of getting up and running with installation profiles that much easier.

This also ties in with ongoing work to make installation profiles - and their component features - more reusable. As we learned during our VoiceBox build, creating reusable features for install profiles is easier said than done.

This is where today's news about the acquisition of two of the most popular installation profiles - Managing News and Open Atrium - becomes relevant. Managing News and Open Atrium were built by Development Seed, and were acquired by Phase2; both of these shops have an amazing track record of high quality work, and of contributing code back to the community. This acquisition represents a significant investment in install profiles, which means that the tools to build and maintain them will likely be improving in Drupal 7 and beyond. It also sets up an opportunity to further the conversation about what is needed to take installation profiles to the next level. Given that we have been building and distributing pre-configured sites since 2005, we're looking forward to continuing our work in this space.

So, as we continue to look at the process of building and maintaining web sites, using installation profiles takes the sting out of the initial site build. Reusable features have the potential to allow useful functionality to be built once, and reused in multiple places.

Automated provisioning scripts take the sting out of building a secure server infrastructure. For many sites, using the advantages of cloud-based hosting takes the sting out of scaling a site.

To be clear, both on their own and collectively, none of these approaches are silver bullets, and running a moderately trafficked web site still requires maintenance time and technical expertise. But, these approaches make running a successful web site more accessible to more people, and will allow greater numbers of people and organizations to get more done with less work and less cost.

Image Credit: "Atrium" taken by Schrottie, published under an Attribution-NonCommercial-ShareAlike license.

NOTE: We do not have any type of sales/affiliate relationship with Linode (or any other hosting company); in our experience, affiliate agreements destroy the credibility and therefore the usefulness of any recommendation. We are, however, using Linode for several internal projects. If/when we discover other options that offer similar value, we'll definitely talk about them, and probably start using them.

Open Content, Where You Want It, In A Space You Control

In the past, we have talked a fair amount about open content. Done correctly, open content has the potential to support meaningful change within education. A significant amount of open content exists on the web; much of this content, however, is not immediately reusable.

In this post and the accompanying screencast, we break down a system that can be used for authoring, redistributing, remixing, and collaborating on open content. The site is customized on top of VoiceBox, and all of the components used to build it are freely available. I'll be presenting on this topic in more detail at Educon in a couple weeks.

This system has several differences from other systems that support open content, but one major difference merits additional notice: it is designed to be porous, and it is designed to be installed and run by the people or organization creating the content. It can be as public or as private as you want it to be, but that decision is left up to individual content authors, as opposed to the people building the content silo.

Overview

Within the site, people can:

  • Create and join groups; groups are workspaces that can be fully or partially private.
  • Author notes; notes are the main form of content within the site. A person creating a lesson would build it by creating notes.
  • Start discussions; discussions occur within groups, and provide a means to discuss areas of general interest, or to discuss a specific note.
  • Offer feedback; feedback is private between the person receiving and the person offering feedback.

Notes can be created as individual pages, or organized hierarchically into multiple page books. Each book can be accessed in print-friendly format, and each book generates its own RSS feed that can be used to import the book outside the site.

In conjunction with a service like BookBrewer, this site can be used by schools or other organizations to create, distribute, and sell eBooks. Also, given that this site is built on Drupal, we just might write some code that allows Views to output content in ePub format.

Video Breakdown, Education Version

Use the minute marks to skip around the video to find the sections that interest you most.

Creating Notes | 0:15

The opening section breaks down how to create notes, and how to shift pages within a book. Notes can contain a variety of different media, and in this section we talk briefly about embedding images, audio, video, and other files within your notes.

Groups | 4:15

This section describes how to use groups to work collaboratively. It shows how to track activity that has occurred within groups, provides an overview of group home pages, and shows the different ways that users can choose to share their content.

To see an example of how to create a group, skip ahead to the 15:00 mark of the video.

Discussions | 7:10

This section shows how to use discussions to collaborate within the site. Discussions can be tied to a specific note, or more general within groups.

Feedback | 9:30

Feedback is a private communication between a reviewer and an author. For example, a teacher can provide private feedback to a student, or a department head can provide feedback and evaluation to an instructor in their department.

Moving and Copying Content | 12:30

In this section, we show how people can work with other people within the site, and how work can be distributed outside of the site. We highlight how to access the RSS feed, and print-friendly format of each book.

For people working in the site, we cover how to copy books between groups to allow people to derive copies and remix content without disturbing the work of their peers

Video Breakdown, Drupal Version

This breakdown gives a high-level overview of how the functionality was created. As mentioned earlier, the site build is based on VoiceBox. The main additions within the Open Content site include the ability to create multiple page books (leveraging core Book module and the contrib Book Manager) and the ability to form working groups (using Organic Groups).

Creating Notes | 0:15

Notes are created using CCK, and the Filefield, Imagefield, and Embedded Media Field modules. With the exception of the embedded media field, this setup is identical to what we are using on VoiceBox. Audio and video nodes are formatted using SWF Tools, and images are displayed using LightBox2. We will likely be switching to ColorBox in Drupal 7.

Groups | 4:15

The groups functionality is created using Organic Groups. The additional access control options for groups are created using the OG Access Roles module.

Discussions | 7:10

Discussions are structurally identical to Notes, with the exception of a nodereference that allows a discussion to refer back to a specific note. The nodereference field is populated using the Node Reference URL Widget module, and the link to create the actual discussion is created via Views.

Feedback | 9:30

Like discussions, feedback nodes use a nodereference to point back to a specific note. Additionally, feedback nodes make use of the Node Access User Reference module to control access to the feedback node.

Moving and Copying Content | 12:30

The ability to copy books is done via the Book Copy module.

Additionally, when books are placed into a group, the group affiliation and the level of access control is kept in synch throughout the book tree via the Book Manager module.

The RSS feeds for books are generated via the Views module, and the print-friendly format comes from the core Book module.

Conclusion

If there any follow up questions, please feel free to ask them in the comments.

Community Media and Grassroots Outreach With VoiceBox

Many moons ago, we announced that our proposal for the Knight Drupal Initiative had been funded.

Yesterday, we pulled back the curtain: the alpha release of VoiceBox is now ready for download.

The site was designed to support community and grassroots media organizations. Out of the box, it supports the following features:

VoiceBox logo
  • Customizable user profiles;
  • Discussions with embedded images, audio, and video;
  • Publishing pictures, audio, and video via mobile devices and/or email;
  • Content aggregation via rss;
  • Remixing, curating, and republishing content within issues;
  • Sharing content to external sites or via email; and
  • Slideshows of featured content on the site homepage and selected landing pages.

As site members create content, they generate their own blog, and each member has a personalized dashboard that gives them an overview of the site.

While the site was designed to support community media and other grassroots organizations, the base functionality can be used in a variety of different contexts. It could easily support a school newspaper, or, with some modifications, it could work as the public-facing site of many organizations that wanted to communicate with their stakeholders and develop a community around their work. In particular, the blend of content aggregation, internal site discussions, repurposing content within issues, and sharing out to external sites via social media helps connect the smaller communities of individual organizations with larger external groups that share a common interest.

The installation profile installs like a regular Drupal install; for people with experience working with Drupal or other open source systems, you can be up and running in around 10 minutes (This video covers the initial installation and some post-installation customizations; it was shot in real time, and it runs about ten minutes).

Additionally, the site is documented, and the documentation includes a set of screencasts.

At the risk of stating the obvious:

The development of this project would not have been possible without the generous funding of the John S. and James L. Knight Foundation. Without their grant support, we would not have been able to devote the necessary development hours required to complete this project.

It's also worth noting that this project could not exist without the countless person-hours that have gone into building Drupal core, and the various contributed modules. In particular, the work of Development Seed deserves special mention. An incomplete list of Dev Seed-powered functionality includes Feeds, Context, and Features; this foundation made much of we built possible.

So, what are you waiting for? Grab the code and get started!

Features, Dependencies, and the UI

A few weeks back, we wrote about our thought process for building features, and how we wanted to make sure that the features achieved two goals:

  • Be part of a fully functioning install profile that "just worked"
  • Be as reusable as possible within VoiceBox variants, or within other sites.

As part of achieving these goals, we created an option within our install profile that allows a user to select the Full or the Minimal install. The full install runs through the complete install, and creates a fully functioning site.

Minimal or Full install option

The minimal install, however, runs through the install profile, and then allows the site builder to enable the selected features they want.

Features are modules; this allows us to set dependencies between features. However, as we built out features based on our site configuration, we realized that the User Interface elements created the largest set of dependencies - and this makes sense, because a good UI collects up the various sets of functionality and presents them to users in an intuitive, coherent way. To account for this, we split out our UI into separate features, so someone doing a minimal install can choose to use these defaults, or build their own UI.

A graph of the dependencies shows this better than words can explain it:

  • Dependency tree of VoiceBox without any UI (click the image to see a full-size version):

    Feature dependencies with no UI

  • Dependency tree of VoiceBox with the UI features included (click the image to see a full-size version):

    Features dependencies, including the UI

    The elements pictured in blue are the UI-related features; within the site, they expose functionality to end users - this is also known as "making things useful or usable."

    Within the features, UI elements create a higher level of interdependence.

We are running through the last few rounds of testing here, and barring any nasty last-minute surprises, we should have a release of VoiceBox by tomorrow. Our features server, which will be launched with the site, will allow people to get the entire install as a package, or the bundled features individually.

An Incomplete History of Sexism In Drupal

So here's the thing. I'm proud to be a member of this community. The Drupal community contains some of the smartest, kindest, most generous people I have ever had the privilege of meeting. I don't think I'm overstating anything to say that my involvement in the Drupal community has altered the trajectory of my life. I have learned more in my six years of work within and around this community than at any other period of my life. It's an amazing place.

And that is why I'd like to see us do better. We can always rationalize away the things we don't like, or find ways to justify things that are distasteful.

But if you see something that feels wrong, stop and ask questions. Realize that you will need to have the same conversation, repeatedly. Realize that people will get mad at you and blame you for bringing it up. Realize that in the process of having conversations, you will learn about things that you misunderstand as well. But don't stop having the conversation, because that's how change occurs: one awkward, uncomfortable, unwanted conversation at a time.

Building Features for Install Profiles

When we originally set out to build a set of Features to support the install profile of our Knight Drupal Initiative work, we figured we would use Context to drive the creation of these features.

Contexts within the site

And this almost worked. Almost.

The Challenge

The challenge we faced in working with features involved sorting out the way Features manages dependencies. When you're setting up a feature, and especially when you're setting up a feature based on a context, the dependencies are automatically brought in and generated for you. So, most of the things that the context relies upon are automatically made a part of your feature (and Strongarm can generally grab the rest). This is great if every single one of your contexts is a completely standalone item. But, if anything in your context contains (for example) a UI element that connects over and exposes functionality that is contained within another context, then you will have two features with overlapping dependencies, and this creates conflicts.

The Goals

At the outset, we had four main goals in setting up our install profile, and the related features used within the profile.

  • Create a site that someone could install and start using;
  • For site maintainers/non-developer admins, make features easy and intuitive to use by creating a logical set of dependencies; aka, features should contain modular sets of functionality, and be as small and as lightweight as possible;
  • Retain the ability to use features to track changes in site config over time. One of the huge benefits of Features is the ability to track changes that you make in config; using the Diff module brings more of the awesomesauce, and here at FunnyMonkey we are serious about the awesomesauce;
  • Make features that are as reusable as possible.

Some Other Things That Didn't Work

Briefly, we contemplated shipping an install profile with one enormous feature. Technically, this works, and the install would have been simple, but the maintainability of this arrangement could potentially get complicated over time. Additionally, one feature that holds everything is not particularly reusable, and it doesn't reveal a clean site architecture through a set of clearly defined and managed dependencies. So, while this would have worked, and is a viable approach in some use cases, it didn't align with our goals.

The next option we considered was to use Features to define functionality, and document how to use the Context module to control block visibility and create a coherent UI to connect the various corners of the site. This would have achieved our goal of making reusable features, but it would have required non-technical users to interact with the Context UI in order to get the most from the site. Given that one of the goals of the entire Knight Drupal Initiative is lowering the barrier to entry for newer or novice users, this approach didn't seem viable either.

Hey! You Put Your Feature In My Module!

One of the many cool things about features is that it pushes config to code; to put it another way, it creates a module from config options. Like modules, features can declare dependencies. So, with this in mind, we exported some initial features that contained the overlapping dependencies as defined above. Then, we edited the exported features so that they contained just the components we wanted them to have - this process included removing the overlaps, and maintaining dependencies on both modules and other features. This allowed us to create some base features that contain the central keys to delivering the functionality - things like content types, imagecache settings, fields, etc. Then, we created some extras features that contain - for example - various views, flags, and other mechanisms used to organize information on the site. Finally, we created UI features that contain the contexts and the reactions that display specific blocks on specific pages.

As we worked on the initial site build - and the subsequent revisions of that build prior to building out features - we also paid careful attention to tags, and to maintaining some consistency with how we tagged and organized our views and contexts. This meta-organization of the building blocks within the site helped us as we got down to organizing the build into features and an install profile.

To get a sense of how this comes together, look at the screenshot below.

The UI feature

This page is pulled together via multiple different features, and they are tied together with a final UI feature.

Using this approach, we are able to meet all of our goals as defined above. Additionally, by separating the functionality from the UI that exposes that functionality, we provide more flexibility for people to use whatever means they want to display content. On our site, we use Context to control block visibility and other display settings. However, someone else could just as easily leave our UI features turned off and use Panels, and/or Drupal's core block visibility settings.

For another example, the home page of the site ships with a slide show, and content displayed within vertical tabs (the base theme we are using for this site is Hexagon; there is some real loveliness in there, but that's a topic for another post). This homepage is generated via a UI feature; if someone wants this UI, they can turn it on. Or, people can build out swappable home page UI features that build on the underlying components, and manage these changes via the features UI.

Homepage slideshow, delivered via a UI feature

The down side of this approach, of course, is that you can't use the Features UI to build your feature, and the process of defining dependencies manually requires some quality time with Strongarm and the variables table.

variables table exportables

Ideally, we could control or manually override dependencies using the Features UI, but patching Features to provide manual overrides of dependencies is no small task, and would likely come at the expense of usability.

By treating features like the modules they are, and by setting up dependencies between them, we can create small, reusable building blocks that retain the maintainability of features created via the standard Features UI. We are getting ready to release our install profile that incorporates this method of building and maintaining features; we still have some testing to do to make sure that we haven't missed or overlooked anything. And with that said, there are likely other ways of solving this, and we would love to hear about them.

Syndicate content