Best Training Ever: Internal Drupal Hackathon at Goodwin College

I just wrapped up a week-long training with my development team today and this was the best training ever: we eschewed the traditional bring-in-the-expert-to-drone-over-some-powerpoint model and styled our week-long training as a hackathon instead.

There were six developers from my team and two from another department all looking to learn how to use Drupal for different projects. These are real projects with real (internal) customers and we all thought that Drupal might be the right tool for the job, but none of us knew how to use it very well. Some of us had played around with it, but some of us hadn't. For one person it was their first time working with a cms. For me it was my second time getting my hands on a drupal instance.

We didn't know how to develop for it as a platform and we didn't know how to install and configure it so that it would be a reliable and secure content management system for our customers. We didn't know how we would set up our dev, test and production workflows, and we just plain didn't know where to begin wading through the sea of modules to find the right tools for our specific needs. Our boss hired an outside consultant to join us for the week to either teach us or work along side us for the week, whatever we needed. We really lucked out getting Tim Plunkett from Zivtech--this guy is an active contributor to many of the Drupal modules that we ended up needing. He knows his stuff and he was excellent at not only teaching us what we needed to know but also teaching us how to find answers when it comes to Drupal and its many modules.

Why was it so awesome?

I have never, ever, learned so much in such a short period of time. It was so efficient. I felt like I was in flow most of the time. I felt that rare electric feeling like the whole room was in flow, like we have never been as a group.

Learn by doing--it's how I learn best; it's how a lot of us learn best. Even better was that because we were all learning different things at different times I had opportunities not only to learn but to teach what I just learned to someone else. This cemented my learning, in a matter of hours, not weeks. On that note, I am sure, SURE, that what I accomplished on the second day would have taken me at least two weeks on my own. That feels amazing.

Here's how it worked

For the most part it was one (or more) project per person, but I was actually on a team with one other developer because our project has a deadline of April 16th (just over a week after the hackathon--ambitious!).

Here were some of our projects:

• Porting of some static html to CMS with theming (this was mine--with the short deadline), laying groundwork for future massive site migration in to CMS
  
• Porting a wiki-based site to Drupal to take advantage of better add-on functionality
  
• Q&A site for teachers (replacing existing software)
  
• Faceted search/browse prototype for a customer with a ton of diverse but related content
  
• Hardware inventory/maintenance history logging system for sysadmins
  
• Massive open online course platform
  
• Site for alum-teachers to share videos and other resources with each other

Ahead of time

Ahead of time we set up a wiki page to keep track of the projects and all of the Drupal questions we had going in to the week. I set up a rough agenda that basically said on the first day we each talk about what we are going to do during the week, then each day we work and stop in the middle to check in (or show and tell) and then on the last day we have final presentations. That's it. I wanted to keep it simple and informal so that we could get as much done as possible. We got some budget to buy snacks and pizza for the week and I found us a room that we could use each day for a whole week (surprisingly difficult!). My colleagues and I agreed that it was important that we get away from our desks if we were going to be able to really focus on learning and getting deep in to Drupal quickly.

The week before I sketched out a layout for the room, a rectangle of tables, with enough room so that the consultant or other people could hop in between people. We sent out invites to our internal customers, asking them to stop by for a check in or to give us some feedback. The Friday before my colleagues helped to set up the room, with my colleague Amir making sure that we had things like an ethernet switch, lots of ethernet cables and power strips and even laptop locks. We checked on things like spare monitors, mice and keyboards, which were conveniently around the corner in the main IT office if we wanted to borrow them.

We checked in with Tim, our consultant, to pick up some best practices for getting our dev environments ready. He suggested things like setting up a space in source control for each project and having php, drupal and drush installed and in the right versions.

The hackathon week began

Once we were all there and we got past the initial introductions and started working it was like magic. I was never stuck, wrestling with a bug or a mystery. Either Tim had the answer, or could find the answer or one of my colleagues had it. When enough of us wanted to see something Tim would do a demo for the group and then we would get back to work. People would bounce around the room from time to time, sharing things they had learned and asking questions. We worked like crazy all day and I never had that feeling in the afternoon like I needed a nap--I was actually energized at the end of the day. (I am exhausted now, but it's a great exhaustion, like after an amazing long run.)

We did check-ins each afternoon, except Wednesday we did a mid-week show and tell just to share what we had learned and accomplished so far. This helped us identify people who had already done the things we needed to figure out how to do. I wrote the first module on Tuesday. Well, it was mostly Tim telling me exactly the words to type and what files to create, but it was a very effective tutorial on module building and a first glimpse for me in to some of the APIs and how Drupal works. On Thursday I learned features and had packaged up my work from the first half of the week in to a feature that could be checked in and pushed out to the other development instances.

About mid-week we realized that a lot of our projects were for different parts the same internal customer's (massive) website and so they should actually live in one big Drupal instance. That required a new project, which was the multi-project multi-developer infrastructure and workflow project. Prior to that we had all been playing in our own sandboxes. By the end of the week we had a solution figured out and the development layer set up so that we could develop features and push them out to each other and keep our development instances in sync. Amir even had a prototype of a staging instance and a workflow for deploying up from development with a plan for how that would work for production too.

In the end we covered just about everything that we had hoped to cover and in a lot more depth than I had imagined possible. We made real progress on real projects and we know a lot of how to do what we have left. Plus, we have learned how to figure out the rest.

So what's next?

At the very end of the event I led the team in a retrospective and almost everyone mentioned that we should do this again. There are a few things that we might do differently, like nail down or narrow down our goals to get even deeper and further along on fewer things or set up a better way to organize common questions and answers that have come up so that we can more efficiently share the knowledge we have gained between us. On the whole everyone loved it and we hope we get to do this again and again and again.

My question is why can't every week be like this at work? Is your work like this?

Show all blog posts on multiple static pages in WordPress

I am working on a hybrid blog/website project using Wordpress and I had a special request come in from one of my clients; He wanted his home page to be a static page, which is easy to do, but instead of having another page as the blog posts page he wanted two blog posts pages. The sidebar would be different for these pages, but the posts would be the same. His reason for this made sense, so I went about looking for a good way to implement it. Using the Reading settings you can only set one page as the posts page. Plus, I wanted to be able to specify a different layout (really just the sidebar is different) for each page, so I couldn't use the Reading settings for this.

The way I addressed this was to create two Pages. I created the layouts for each page with my iThemes Builder Style Manager plugin. Then I created a new page template, so that the pages would display posts instead of the page content. I took my theme's single.php code and expanded it so that it would show all of the posts plus the static Page's title at the top. I copied the loop code from my theme's single.php file. Below you can see the part that I changed, which is just the top part of the file:

<?php
/**
 * Template Name: Posts Page
 *
 * A custom page template for DataPoints
 *
 */
function render_content() {
?>

<!-- Display the page title -->

<?php if ( have_posts() ) : while ( have_posts() ) : the_post(); ?>
 <div class="post">
  <!-- Display the Title as a link to the Post's permalink. -->
  <h1><?php the_title(); ?></h1>
  <hr/>
 </div>
<?php endwhile; endif; ?>

<!-- Show all of the blog posts -->
<?php query_posts( 'posts_per_page=5' ); ?>
<?php if ( have_posts() ) : ?>
  <div class="loop">
   <?php while ( have_posts() ) : // The Loop ?>
    <div class="loop-content">
     <?php the_post(); ?>
<!-- The rest of this is identical to my theme's single.php file -->
...[SNIP]...

Listing Sub-Pages on your Page in WordPress

I know I'm a WP noob, but this took me about an hour to figure out. Maybe I can save someone else some time.

Most of the code below comes from the WordPress docs, but there was one critical piece missing, $post->ID was returning nothing, even though I was in "the loop". I got around that whole thing once I learned that you can get the page id with get_the_ID().

<?php
// display only this page's children
$my_id = get_the_ID();
$subpages = wp_list_pages("title_li=&echo=0&depth=1&child_of=".$my_id);

if ($subpages) { ?>
<h2>Sub-topics</h2>
<ul>
<?php echo $subpages; ?>
</ul>         
<?php } ?>

I am using WordPress 3.0.1, if that helps anyone.

My First Spring Roo App (two months later)

Getting Started with Spring Roo

When I started working with Roo I think they were on RC 2 and now we are on the full-fledged Release 1.0.0, and I noticed that a lot of the tutorials that got me through those early days are a little outdated since a few common commands and generated output have changed over the past two months. I thought since I borrowed so heavily from the ROOsters before me (thanks BTW, Ben Alex and Stefan Schmidt) I would return the favor and talk about my experience a bit. I haven't gotten around to writing my own step-by step guide, but at least I can tell you more about why I chose Roo and pass along the things I have learned along the way.

I don't remember when I first learned about Spring Roo, but right from the start I was impressed. I cut my Spring teeth working on a few fairly large and complicated Spring webapps for about a year before I met Roo. I did the Springsource training, drank the koolaid; I honestly like working with Spring and I feel at home working in Java, but I did miss the fun and seemingly fast-paced development on the Rails projects from my past. Roo promised to bring back the fun and help to make a lot of the routine work of writing Java code disappear.

I had a little side project that started up in November of 2009; the idea was basically a really simple CRUD app. Three domain objects, some security, done. We all know how requirements do tend to change and multiply (especially when you bring a nice-looking and complete webapp as a demo to the first meeting, thanks to Roo), but this project is under control and I am still using Roo to add functionality (like email sending) and to manage the simple domain objects and their controllers and views.

Gotchas

Oh man, have I fallen into a few holes on my way to getting this app off the ground. Some of them turned out to be ROO bugs (I was working with the release candidates, after all) and others were just boneheaded mistakes on my part. In hopes of helping you avoid the same traps here is my list of gotchas and tips for working in Roo. I'll update these as I get a chance, but here are some of my recent lessons learned:

• If you push in Roo on your domain object so that all of the entity management code is laying around and love to collect all of your imports under something like javax.persistence.* make sure you have an explicit import for javax.persistence.EntityManager. I usually set my minimum pretty low, so that if I optimize imports (ctrl + shift + o in eclipse/STS) then it will use a * instead of listing imports individually if I have more than 10 or so from the same package. That is normally fine, but causes serious problems for your little Roo app. It will yell at you for not returning the right type on your entityManager() method.
• I am using some security on my manager methods and I went to write a test and I wasn't getting the results I expected. It was saying something about "can't parse expression" referring to the Spring EL expression that is used in the PreAuthorize annotation on my manager method. Well, it turns out I was using the wrong kind of authentication token to fake someone being logged in for my tests. Here is the real way to do it:
  
  

  I call this from my test:
  

  private void addResearcherToSecurityContext(Researcher researcher, boolean admin) {
       Collection authorities = researcher.getAuthorities();
       if (admin) {
           authorities.add(new GrantedAuthorityImpl("ROLE_ADMIN"));
       }
       TestingAuthenticationToken authentication = new TestingAuthenticationToken(researcher, researcher.getPassword(), authorities.toArray(new GrantedAuthority[authorities.size()]));
       SecurityContextHolder.getContext().setAuthentication(authentication);
  }
  

  
  and put this in my test application context:
  

  <bean class="org.springframework.security.authentication.TestingAuthenticationProvider" id="testAuthenticationProvider" />
  

  
  and put this in my test application security context:
  

  <authentication-manager alias="authenticationManager">
  <authentication-provider  ref="testAuthenticationProvider" />
  </authentication-manager> 
  

  
  Simple, huh? It really is. You should know that my Researcher object extends UserDetails and that is what I use for authentication. Now I can write methods where I try to update someone else's account or do admin-y things as a non-admin and make sure that I get an access denied exception and all my data is safe.
  
  
• This is more of a spring/hibernate thing than a Roo gotcha, but I'm going to put it here anyway because it just happened to me again... If you have a manager or, in this case, a test, and all of the methods are transactional. Don't annotate each method with @Transactional because inevitably you are going forget one time and then sit there running your test over and over wondering why it is saying "org.hibernate.SessionException: Session is closed!". You can stick the @Transactional on your class and then you are all set, no matter how many methods you add.
• Don't forget: the forums and jira are your friends. There is also a seemingly constantly growing list of Roo links and resources on the forum

Pair Exchange Program

I just finished the first ever pair programming exchange program!

It all started when I was at SDTConf back in October. I don't remember the exact conversation, but the question was generally if software development as a craft takes two programmers pairing together to pass on and practice the craft then how will it ever scale?

Someone jokingly suggested that Corey Haines should get a bigger car--Corey travels all over the place pairing with people and promoting the craft of software development. One idea that I think Corey actually proposed was the idea of a pair exchange.

The idea is simple: we can't all get Corey Haines or another big name in the XP, Agile, Craftsman, etc. communities to come and pair with us, but there are lots of smart people in every town. If you two are both passionate about practicing and improving your skills then why not use each other as a resource? Everyone has a vacation day to spare, take a day and go pair. If your boss will let you, bring your pair partner to work.

I know a lot of developers in Philadelphia and I had no trouble thinking of the perfect programmer to pilot my pair exchange program with. Woody Zenfell III is not only one of the smartest and most professional programmers I have ever worked with, but we make a killer pair. It is like our brains are wired almost exactly oppositely, but we get along really well. I like that he is fun to work with and we get a lot of good work done together. Plus, I know his company already and I know that he is on a team of 1 and is probably missing working with other developers.

When I first brought this idea up to my manager I had reason to be optimistic. My manager sees the value of pairing, even though my team doesn't really practice it much, and he is very supportive of learning and professional growth. I was thrilled when he agreed to the whole thing, both me going to Woody's office for a day and for Woody to come and pair with me and chat with my team. Woody's boss was also game and we were set!

We decided to do one day at each office, back to back. Since this was our first pair exchange we didn't really know what to expect other than that we would show up, work on some stuff and see what value we get out of it. For me, I knew Woody's code base pretty well, having actually worked on it with him back in our consultant days. For Woody, my team culture and our domain and the code base were all new, so a chunk of time in the morning went to talking about those things. Other than that we just paired; it was awesome. At Woody's place we had some tasks that needed to get done and so we did them. At my place we talked about design for a long time and wrote a test that I was having a hard time writing. Lots of great side-conversations happened with my teammate neighbors. It was fun and we got a lot done.

What did we get out of it?

For this first exchange it seemed like we were for the most part exchanging little tips, tricks and techniques that make us more efficient and effective. Woody, for example, showed me how he is creating dependency graphs (actual pictures) for all of the stored procedures and reports in his system, automatically, so that he can more confidently make changes to them. He is also getting those database elements into source control and refining a test and deployment process for them, which is something I hadn't thought of before, but is really smart given how important those reports are to his company. He also showed me captures in EasyMock, which were exactly what I was always dreaming of but never had because we were still using EasyMock 2.3. In return I showed him ctrl+r in cygwin/linux for searching your history. Then he showed me pushd and popd--that's just so handy.

In addition to tips and tricks we had a lot of conversations about how we do things, like how we work with our users and what process we follow for organizing our work and testing and deployment.

At the end of the second day we took several laps and talked about the experience. We agreed that it was definitely valuable and something we want to do again. We imagined that after a while we will run out of tips and tricks to show each other and then we didn't know what would happen then, but maybe that we could use each other as reinforcements for when something really sticky comes up and we need a fresh perspective. We talked about doing it again quarterly or monthly or more randomly. We aren't sure how our bosses viewed the experiment, and of course they would have to feel like they are getting a good deal out of this for them to allow it to continue. A lot of factors went in to making this experience go smoothly: the fact that we have both paired a lot before and with each other before, that our bosses are agreeable, there are no super-high security requirements that would prevent guests from coming over and peering at the code, etc. I am curious how well it would go if I was pairing with someone I didn't know very well.

I would love to hear about more people who are trying this sort of exchange. If you don't have teammates who pair (or teammates at all) you don't have to miss out on the benefits or pair programming--the discipline and the sharing of new ideas.

Url patterns for servlet mapping

I came across a baffling problem with my servlet mapping recently for my java web application. In this project we generally use extension-based url patterns for servlet mapping. For example:

 <servlet-mapping>
  <servlet-name>myServlet</servlet-name>
  <url-pattern>*.html</url-pattern>
 </servlet-mapping>

This means that every request that comes in that ends in ".html" will get passed into my servlet instead of being handled by Tomcat (who would serve it up like a static file). The problem came up when we could not use extension mapping because we had a few .html files that needed to be served up by Tomcat and the rest were dynamically generated by my servlet. But when I tried to use a path-based url pattern it didn't seem to work as I would expect.

Here is the situation: I know that all .html urls that have the word "library" in the path to need to be sent to my servlet and all other .html requests to be handled by Tomcat, so I tried the following:

 <servlet-mapping>
  <servlet-name>myServlet</servlet-name>
  <url-pattern>/library/*</url-pattern>
 </servlet-mapping>

My servlet is expecting requests to come in like "/library/index.html", and it worked with the extension url mapping, but with the path url mapping it didn't. It was like nothing was getting passed into my servlet. I did some searching and I didn't find anything in the Java Servlet Specification or on any blogs or forums that explained why path matching would be so different from extension mapping, until I came across this post where PhilipT in the comments explained how tomcat was stripping out the matched part: "So it seems that the Tomcat default servlet (6.0.16 at least) drops the servlet mapping and will try to find the file by using the remaining path." This made me wonder if that always happens with path matching.

I also learned from that post that you can map urls back to tomcat from your servlets web.xml file. In our case we haven't tweaked tomcats configuration so the default servlet is called "default". I let my servlet handle all .html files and I tell tomcat to handle things that come through with the words "static-content" in the path. Since the order of precedence is exact matches, path matches and then extension matches this works exactly as I want it to.

 <servlet-mapping>
  <servlet-name>myServlet</servlet-name>
  <url-pattern>*.html</url-pattern>
 </servlet-mapping>
 
 <servlet-mapping>
    <servlet-name>default</servlet-name>
    <url-pattern>/static-content/*</url-pattern>
 </servlet-mapping>

The only catch with this solution is that you have to remember to put /static-content/ in front of the path to the static content that you want served up by tomcat. It is possibly confusing because there is no "static-content" directory on the server. To give an example, the html files that need to be served by Tomcat are for the fckeditor that we use in our web application. When you want to include a fckeditor on a page you have to tell the javascript where to find the fckeditor home directory. For our project it is a directory called fckeditor sitting in the war directory for our application. Previously we would just say that the home directory was /fckeditor/ and Tomcat would know to find it in <war directory>/fckeditor. Now that we are using path matching we have to tell the fckeditor javascript that its home directory is /static-content/fckeditor/, even though it is still physically on the server at <war directory>/fckeditor