FLOSS Foundations

This Week

Governance

• Careful comments on fligtar's proposal for changing the AMO sandbox model
• Started looking at the issue of whether we have private mailing lists which shouldn't be private
• Worked with harvey on streamlining procedure for e-signing Committer's Agreements
• Mitchell resolved weekly update meeting move discussion; will now be 11am Monday

Bugzilla

• Summarised where we are so far
• Call with justdave about b.m.o. update procedure, technical feasibility of various plans

Other

• Watched Firefox 3.5 release process with admiration
• Updated Firefox Language coverage data for 3.5 and reblogged it
• Finished OpenTech presentation (nearly)
• Mentored SoC students:
  • Pedro has written a plan which looks very good, and will start implementation this weekend
  • Am working with Seulki to redefine the scope, as the original idea is looking difficult

Next Week

• OpenTech 2009 presentation on the Open Internet (Saturday)
• Firefox Launch Party, London (Monday)
• Make sure sweepstake winners get their prizes
• Wednesday off to prepare summer Bible studies
• Try and unblock some issues now people are not swamped with Firefox 3.5

Vivek Kundra, the CIO of the United States, unveiled the new IT spending dashboards at usaspending.gov earlier this week. Tim O'Reilly has all the details in his blog post titled Radical transparency: the new federal IT dashboard. In short, the dashboards are designed to help CIOs of individual government agencies get a handle on the effectiveness of government IT spending. The site was built with Drupal.

This is a fundamental change in the way government is going to be run, and it is great to see Drupal play a small role in that. Great stuff!

From a Google search inspired by an idea I got reading Scott Berkun's Why panel sessions suck (and how to fix them):

Why has no-one commercialized this? It's the obvious solution to the "pass the microphone" problem at panels, talks and other audience-participation events.

The GNOME press contact alias got a mail last weekend from Sam Varghese asking about the possibility of new Mono applications being added to GNOME 3.0, and I answered it. I didn’t think much about it at the time, but I see now that the reason Sam was asking was because of Richard Stallman’s recent warnings about Mono - Sam’s article has since appeared with the ominous looking title “GNOME 3.0 may have more Mono apps“. And indeed it may. It may also have more alien technology, we’re not sure yet. We’re still working on an agreement with the DoD to get access to the alien craft in Fort Knox.

Anyway - that aside, Richard’s position is that it’s dangerous to include Mono to the point where removing it is difficult, should that become necessary to legally distribute your software. On the surface, I agree. But he goes a little further, saying that since it is dangerous to depend on Mono, we should actively discourage its use. And on this point, we disagree.

I’m not arguing that we should encourage its use either, but I fundamentally disagree with discouraging someone from pursuing a technology choice because of the threat of patents. In this particular case, the law is an ass. The patent system in the United States is out of control and dysfunctional, and it is bringing the rest of the world down with it. The time has come to take a stand and say “We don’t care about patents. We’re just not going to think about them. Sue us if you want.”

The healthy thing to do now would be to provoke a test case of the US patent system. Take advantage of one of the many cease & desist letters that get sent out for vacuous patented technology to make a case against the US PTO’s policy pertaining to software and business process patents. Run an “implement your favourite stupid patent as free software” competition.

In all of the projects that I have been involved in over the years, patent fears have had a negative affect on developer productivity and morale. In the GIMP, we struggled with patent issues related to compression algorithms for GIF and TIFF, colour management, and for some plug-ins. In GNOME, it’s been Mono mostly, but also MP3, and related (and unrelated) issues have handicapped basic functionality like playing DVDs for years. In Openwengo, the area of audio and video codecs is mined with patent restrictions, including the popular codecs G729 and H264 among others.

What could we have achieved if standards bodies had a patent pledge as part of their standardisation process, and released reference implementations under an artistic licence? How much further along would we be if cryptography, filesystems, codecs and data compression weren’t so heavily handicapped by patents? Or if we’d just ignored the patents and created clean-room implementations of these patented technologies?

That’s what I believe we need to do. Ignore the patent system completely. I believe strongly in respecting licencing requirements related to third party products and developer packs. I think it’s reasonable to respect people’s trademarks and trade secrets. But having respect for patents, and the patent system, is ridiculous. Let a thousand flowers bloom, and let the chips fall where they may.

So if you want to write a killer app in Mono, then don’t let anyone tell you otherwise. If you build it, they will come.

Where's the action these days? I'm currently on Dopplr but have had TripIt invitations. Where are most people in the Mozilla community? A particular one, or both? I hear TripIt parses your emails from Expedia (which Mozilla uses) and other such places, which would be a time-saver. But migrating everything over would be a pain. I guess this is my first experience of potential social networking data migration anxiety. Open data FTW!

Please declare interests such as board seats or shares ;-)

I finally decided to move my blog. kdedevelopers.org has served me well, but now I want some more features. Blogger provides some killer features, such as using my own domain, blogging by email, or the powerful comment system. So from now on you'll find my blog at blog.cornelius-schumacher.de. See you there.

Feel the Shiretoko Shock!

So what am I personally most excited about in the new Firefox? Well, I have no uses for Private Browsing Mode - I think porn is a terrible corruption of God's design for sexual relationships, my personal medical condition that I research is by no means a secret, and I don't share my computer with anyone anyway. The security UI improvements are cool and well worth having, but I'm not going to get phished any time soon. It's wonderful that we're now in even more languages, but I don't speak any of them!

So really, I'm just waiting for sites to take the excellent technical and standards changes and build cool new stuff I can use. Web designers, what are you waiting for? :-)

Yesterday we took the beta-wraps off of Acquia Search, and I followed up with a post about why Acquia Search matters for site visitors. We're still having some good discussions in the comments and the Twitter-sphere, but today I want to talk a bit more about the technical details. How does Acquia Search work, what does our infrastructure look like, and why is it a great deal for site owners?

Acquia Search is a hosted search service based on the Software as a Service (SaaS) model. The way it works is that Drupal sites push their content to the search servers hosted by Acquia. We index the content, build an index, and handle search queries. We provide the search results, facets, and content recommendations to your Drupal site over the network.

Your site's data is protected in transit by SSL and by HMAC authentication in the Acquia Network. Plain english? The data is encrypted so anyone snooping in the middle can't read it and the request is authenticated which means that the Acquia Network knows you sent the request you claimed to, and you know that messages received from the network are legitimate.

Acquia Search is built using the Open Source Lucene and Solr distributions from the Apache project. If you want to install, run and maintain Lucene and Solr yourself, and you have the resources to do so, you can. All the code, including our contributions to the Apache Solr integration modules for Drupal, are available as Open Source.

However, many organizations simply lack the Java expertise to deploy, manage and scale Java applications -- or their hosting environment may not accommodate it. Because Acquia Search is a hosted service, it takes away the burden of installation, configuration, and operational duties to keep the software fast, secure and up-to-date. That's our job.

As a reference, we've spent the last 9 months developing Acquia Search with the equivalent of three full-time employees. This also included setting up a billing system, integrating our support system, connecting it to the Acquia Network, performance testing and tuning, and more. Other Acquians helped out with the infrastructure, quality assurance, product management, design, and documentation. It was a non-trivial amount of work.

The result of these efforts is that we can launch any number of Solr farms on Amazon EC2. For high-performance and high-availability, each farm has a master Solr server and one or more slave Solr servers. A load balancer pushes content changes to the master Solr server, which are replicated by the slave servers. The load balancer makes sure that most regular search queries are done against the slave servers. Because multiple servers can handle your site's search requests, Acquia Search is fast and can scale, but it also means that Acquia Search is very robust because it can survive a server failure. As I wrote yesterday, Acquia Search is faster that Drupal's built-in search -- especially for large sites.

In most scenarios, several Drupal sites share a single Solr farm -- by sharing resources, we can offer a high-performance and high-availability search solution to small sites at relative cheap price point. For really big sites, we can provision a dedicated farm and scale out Solr so that it can handle millions of search queries.

Once you begin to use our search service you'll be able to disable Drupal's built-in core search. When you do this you reduce the amount of memory and processing power needed by your own infrastructure. As we've learned with big sites like drupal.org, Drupal's built-in search can bring a large site to its knees. With Acquia Search, you can avoid the drain.

On the front-end, we made significant contributions to the Apache Solr Search Integration modules on drupal.org. We helped add new features, improve the usability, and iron out a legion of bugs that cropped up during the beta period. The top-3 most active maintainers of the Apache Solr module are all Acquia employees, respectively Peter Wolanin, Robert Douglass, and Jacob Singh. As a result, Peter, Robert and Jacob are sometimes referred to as Acquia's three Apache Solr Musketeers.

All things combined, Acquia Search makes it staggeringly simple and low-cost to get better search on your site. You can get started in minutes and you don't have to worry about installing, upgrading, monitoring, or scaling the software. In short, we built an enterprise-quality, highly-available, secure, scalable, and fast indexing search solution that we believe Drupal was missing -- especially for the enterprise.

As of 3pm BST this afternoon, Firefox 3.5 has been unleashed on the world! Either do Help | Check For Updates or download it here. Various community members have recorded a video where they get excited about some of the new great stuff. If you are using an older browser, you may need to upgrade before you can view it :-)

All geeks in London who are pleased about this are invited to the Firefox 3.5 Launch Party London, next Monday the 6th of July from 7pm at the Shooting Star pub in Middlesex Street, near Liverpool Street station. Sign up on Upcoming if you plan to be there. We have a big screen, so bring cool stuff to demo if you have it, and a party mood. :-)

Why will the visitors of your site care about Acquia Search? For a while now, I have Acquia Search installed on my personal site. To understand what Acquia Search can do for your site, have a look at what it has done for my site. While I have a very simple Drupal site, you should be able to experience some of the benefits of Acquia Search.

For example, search for "Drupal" on my site (use the search widget in the sidebar) and you can see the facets that allow you to filter the results by topic, location and industry. Using these facets, it should be pretty easy to find all the Fortune 500 Drupal sites that I blogged about in 2009, for example. Facets make search faster, making it very easy for your visitor to drill into results and to find what they are looking for.

Acquia Search makes search easier because it is built on the principles of progressive disclosure. Instead of showing the visitor an initial page with lots of complicated options (see Drupal's advanced search options that almost no one uses), the facets are only shown after the initial search query. Plus, and this is really cool, facets are dynamically generated based on the search keywords. As such, they are relevant to what you're searching for.

Acquia Search provides a more powerful search because it is based on the renowned open source Lucene and Solr technologies from the Apache project. Not only do they sport better search algorithms, advanced content normalization, and a "did you mean?" feature, they also come with other great features such as word stemming, document search, range queries and more.

My favorite feature of Acquia Search, at least for use on this blog, is the "more like this" feature -- on node pages you can ask Acquia Search to suggest related content. I have been using it on my site for a while (see the block in the sidebar), and it has helped to keep visitors on my site longer. I occasionally find myself getting side-tracked by the "related links" -- it is a great way to re-discover old posts.

Last but not least, our new service makes for better performance. We performed tests of searches on a Drupal site with over 10,000 nodes of content using a 3.2Ghz dual core server with 1.7 GB of RAM. With Acquia Search results were displayed in less than half a second, whereas the same results served from Drupal's built-in search took anywhere from 1.5 to 7.7 seconds. On the web, faster is better.

That makes for a lot of good reasons why the visitors of your site might care about Acquia Search. Tomorrow, I plan to write a more technical blog post about how Acquia Search works, how we made it that fast, and why it matters to site administrators (instead of site visitors). In the mean time, I recommend that you play around with the search feature on my site or that you sign up for a trial subscription. Have fun!

Two days of KDE Wiki Meeting are over. Danimo, Frank, Lydia, Dominik, Milian, Thorsten and me met in Berlin with the goal to get some more structure into the KDE Wikis and provide a plan for the future, where to put content. I'm happy to say that we accomplished this mission.

While we have TechBase for high quality technical documentation for a while, and the corresponding UserBase for end-user information since last year's Akademy, we were still missing a proper place for community content, especially for content which is mostly community internal, of more transient
nature, or just not finished yet. The idea to create a dedicated Wiki for this community content was floating around since a while, and now we created it at community.kde.org.

To make it clear which content belongs where, we created a mission statement, which gives clear guidance about which Wiki serves which purpose. You'll find it at wiki.kde.org in a few days. The basic idea is that userbase.kde.org provides end-user information, techbase.kde.org contains high-quality technical content for third party developers, distributors, and system administrators, while community.kde.org acts as a collaboration space for the community.

Actually community.kde.org already existed. It contained the charter of the community working group. But to keep things short and to the point we decided against creating another base, but go with the logical and short community.kde.org domain. The charter of the CWG will find a new home on the KDE e.V. web site.

With the creation of community.kde.org we can also shut down at least two places where community content ended up due to lack of a proper home. We'll shut down the old Wiki, which was available under wiki.kde.org, but whose content wasn't that well maintained, and which didn't fit too well in KDE's infrastructure because of technical reasons. We'll also move all the pages which piled up under the Projects directory on techbase, but in almost all cases didn't really belong there and also didn't match the quality requirements of being polished content targeted at technical people who aren't necessarily familiar with the community. Most of these pages find a proper home on community.kde.org, though.

In addition to the general cleanup and structuring we also worked on some improvements of the existing Mediawiki installation. Danimo replaced the OpenID login UI by a much more usable version, Milian finally managed to get rid of the annoying horizontal scrollbars inside the page on code samples, and we also discussed some more improvements, like the intensified use of templates and the introduction of a way to rate and classify documents on the Wiki to indicate their quality and make it more obvious what needs more work.

As a side track, we had an interesting discussion with some Tiki developers. They have an amazingly powerful and feature rich system, which would
be able to solve some of the problems, we still have with our Wikis, such as translation infrastructure. For now we decided for the sake of consistency and simplicity to stay with the current Mediawiki installation, but maybe Tiki is an option in the future.

Working on the Wikis was fun and satisfying because we got some concrete results, which will simplify maintaining KDE web content in the future. But besides all the work we also didn't forget to relax with a great dinner at a Chinese restaurant at Berlin-Adlershof, enjoying a great buffet, including cheese cake for dessert.

Thanks go to the KDE e.V. and Qt Software for supporting the meeting.

It's a big day for us at Acquia. We finally took the beta-wraps off of Acquia Search, and made it available commercially as part of the Acquia Network. Thanks to the 250+ beta testers who helped make our hosted search service fit for use in production environments, including Brightcove, JackBe Developer Community, P-O-P Design, Wide Divots and others.

We used the beta period to look at the usage statistics, costs, and to talk to a lot of beta users to figure out the best pricing model for this service. We decided on the following:

Acquia Search is included for no additional cost in every Acquia Network subscription. Basic and Professional subscribers have one "search slice" and Enterprise subscribers have five "search slices". A slice includes the processing power to index your site, to do index updates, to store your index, and to process your site visitors' search queries. Each slice includes 10MB of indexing space - enough for a site with between 1,000 and 2,000 nodes. Customers who exceed the level included with their subscription may purchase additional slices. A ten-slice extension package costs an additional $1,000/year, and will cover an additional 10,000 - 20,000 nodes in an index of 100MB.

For my personal blog, which has about 900 nodes at the time of this writing, a Basic Acquia Network subscription ($349 USD/year) would give me all the benefits of Acquia Search, plus all the other Acquia Network services.

For some of you, this might sound like a lot of money, but we believe you get a lot of value in return. In my next couple of blog posts, I plan to outline the benefits of Acquia Search to your site visitors and to Drupal site administrators. Stay tuned!

Update: fixed a bug where I had confused Maltese and Malayalam. Firefox's numbers improve further :-)

The languages to be included in Firefox 3.5 are now confirmed, and Microsoft has just released the last 5 additional languages for IE 8 on XP they said were pending. That's three months after the original release. I guess they still need to learn from our team on how to simultaneously ship them all at once ;-P (And it's worth noting quite a few of their languages are still Vista and/or 32-bit only. Should we count a language if it's not available on their most common OS?)

Anyway, it's time for an update on our plans for world domination. A reminder for those joining the party: I have a spreadsheet which combines the Internet population of a country with its first-language-spoken percentages to try and get an accurate idea of what languages the world Internet population speaks.

The headline is that of the latest releases of all the browser vendors, Firefox now has the highest percentage of world Internet population first-language coverage - 95.7%:

Browser	Count	Percentage
Firefox 3.0	59	92.4%
Firefox 3.5	68	95.7%
Opera 9.6	35	84.8%
IE 7	77	98.3%
IE 8	57	94.6%
Chrome	39	91.6%

The key change from the last release of the data is that I've got much more up-to-date figures for Internet population from InternetWorldStats (thanks very much to them). There are 28.5% more people on the net than in the last release of the statistics - an extra 300 million people. Almost every country's net population has gone up, but they've gone up by different amounts, and so that has had a knock-on effect on the relative importance of different languages, and of the percentages of the world covered. So, for example, Firefox 3.0 "dropped" more than one percentage point from 93.5% to 92.4%, despite still covering the same languages it always did! As more of the developing world comes online, the goalposts are moving :-)

The stats have their own page, where you can find the spreadsheet, FAQs and a more detailed explanation. A few noteworthy points:

• If you want a current browser in Esperanto, Welsh, Persian, Irish, Galician, Icelandic, Kurdish, Occitan, Romansh or Sinhala, Firefox is the only game in town.
• We are releasing 3.5 in Turkish despite the Turkish localizer breaking his leg. That's dedication.
• Axel says we are looking at taking a Malay localization from Ubuntu. That would be another 0.72%. Seth says Azerbaijani is in progress, and it could be that some friends of build engineer Ben Hearsum are looking at Tagalog. Exciting times :-)

I just lost my browser and my entire desktop, AGAIN, while watching a video. This is an all-too-regular occurrence. Flash is the top cause of crashes on Linux - and has been among the top causes since Firefox 3.

When, oh when, will someone ship a browser I can use that can show me video without needing the Flash plugin?

Firefox 3.5: fonts, speed, privacy enhancements and much more.

Available tomorrow by download or by using Firefox’s “check for updates” feature, which is found in the “Help” menu.

Plus “Shirotoko Shock” — a fun, easy way to be part of the launch!

I've just got back from a trip to the UK and Italy last week for the OSGIS event and the OSGeo Hacking Event. It was a great time of connecting with so many people that it exceeded my expectations.

Before heading up to Nottingham I was able to meet up with Martin Daly from Cadcorp (a GDAL/OGR sponsor) in London. We hadn't met face to face before, so I was glad to get a firsthand chance to see their SIS GIS Desktop product in action. Our very own Mateusz also works there so it was treat to catch up with him as well. Of course he couldn't resist going to Nottingham too and overworking his camera - his flickr feed is always helpful after an event :)

From the word go, Suchith Anand, Mike Jackson and the rest of the University of Nottingham's Centre for Geospatial Science crew had us hopping. I didn't realise it was possible to pack so much into a single day event! Thank you for the opportunity to deliver a keynote there and for lining up so many good speakers and people to meet with the following day as well. Three cheers! I'm sure next year will be even better.

It was a truly international event with close to 20 different countries represented. It was really good to catch up with many of you from all over Europe, I'll be going through my notes this week, but still feel free to drop me if you remember anything in particular we should pursue together.

One recurring theme of interest from several groups and and individuals, was the interest to partner with OSGeo to write case studies relating to their domain. I'm really looking forward to following up on these. If you want to help write, please let me know.

The OSGeo UK Chapter held a meeting as well, with Jo Cook leading the charge. It's always fascinating to see how different each chapter is - each with their own interests, strengths and local challenges. This was only the second time they met...sort of. Last year there were less than a dozen who met up to discuss the idea of forming a chapter. This year it was more like 50 or more people with broad interest from around the UK. Coming from a Canadian, it might seem strange to say that the UK chapter is spread out over large distances, but it is large enough that some more regional local groups are likely to pop up. There were contingents from Scotland and Wales talking about meeting and I know there are some Irish interested as well.

They have high hopes, so if you are the UK, join their list and find a good reason to get together and chat.

(Photos from mlsoskot's photostream under under these terms - good job Mateusz!)

This Week

Governance

• Call with reed to gain lots of useful technical info about possible solutions to various governance questions
• Did groundwork necessary for mozilla.org/hacking/ to redirect to the Developer's Guide on MDC
• Wrote first draft of Committing Rules and Responsibilities and had it reviewed by bsmedberg
• Participated in discussions on moving weekly update meeting

Bugzilla

• Discussion has now been wrapped up or taken to email
• Triaged a few bugs filed against b.m.o. to get a feel for the space
• Need to do more work on the issues list

Other

• Tuesday off to finish moving house
• Wrapped up the Bugzilla 500,000 bug sweepstake
• Reviewed and commented on CSP spec
• Started work on talk for OpenTech 2009
• Updated credits.html with backlog of requests, in preparation for release
• Mentored SoC students
  • Pedro is trying to grok liveHTTPheaders
  • Seulki is working on a plan

Next Week

• Make sure sweepstake winners get their prizes
• Call with David to discuss our OSCON presentation
• Finish OpenTech presentation