playdoh Documentation Release 1.0 Mozilla February 13, 2015 Contents 1 Contents 1.1 Quickstart . . . . . . . . . . . . . . . . . 1.2 Installing Bedrock . . . . . . . . . . . . . 1.3 Vagrant Installation . . . . . . . . . . . . . 1.4 Installing and Learning About the PHP Site 1.5 Localization . . . . . . . . . . . . . . . . 1.6 Developing on Bedrock . . . . . . . . . . 1.7 JavaScript Libraries . . . . . . . . . . . . 1.8 How to contribute . . . . . . . . . . . . . 1.9 Using Grunt . . . . . . . . . . . . . . . . 1.10 Newsletters . . . . . . . . . . . . . . . . . 1.11 Tabzilla . . . . . . . . . . . . . . . . . . . 1.12 Mozilla.UITour . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 3 3 8 12 18 23 25 25 28 29 32 33 i ii playdoh Documentation, Release 1.0 bedrock is the code name of the new mozilla.org. It is bound to be as shiny, awesome, and open sourcy as always. Perhaps even a little more. bedrock is a web application based on Django/Playdoh. Patches are welcome! Feel free to fork and contribute to this project on Github. Contents 1 playdoh Documentation, Release 1.0 2 Contents CHAPTER 1 Contents 1.1 Quickstart You can develop and test bedrock without installing anything locally by using Cloud9, which provides a complete development environment via your browser, including unlimited free public workspaces 1 , which is great for open source projects. Each workspace includes root access to an Ubuntu Docker container, which you can install bedrock and all its depdencies into with the following steps: 1. Fork bedrock on github 2. Sign up or sign in to Cloud9 with your github account 2 3. Create a new workspace from your fork using the “Clone from URL” option with a URL in the format [email protected]:mozilla/bedrock.git but with your username instead of mozilla 4. Once your workspace is ready, click the “Start Editing” button 5. In the bash shell, run the command bin/install-c9 Once the install-c9 script completes, you can click the Run Project button to launch the django development server, which will be accessible on a public URL similar to http://bedrock-c9-username.c9.io 1.2 Installing Bedrock 1.2.1 Installation It’s a simple Playdoh instance, which is a Django project. These instructions assume you have git and pip installed. If you don’t have pip installed, you can install it with easy_install pip. Start by getting the source: $ git clone --recursive git://github.com/mozilla/bedrock.git $ cd bedrock (Make sure you use –recursive) 1 Public means everything in the workspace is world readable; you can also grant write access to specific cloud9 users and collaboratively edit code in your workspace in real time. 2 Github account integration is optional; if you do not wish to give cloud9 access to push to any repo your github account has access, you may wish to use a deploy key or a machine user account. 3 playdoh Documentation, Release 1.0 Important: Because Bedrock uses submodules, it is important not only to use --recursive on the initial clone, but every time you checkout a different branch, to update the submodules with: git submodule update --init --recursive You might want to create a post-checkout hook to do that every time automatically, by putting that command in a file bedrock/.git/hooks/post-checkout. You need to create a virtual environment for Python libraries. Skip the first instruction if you already have virtualenv installed: $ $ $ $ $ pip install virtualenv virtualenv venv source venv/bin/activate pip install -r requirements/compiled.txt pip install -r requirements/dev.txt # # # # # installs create a activate installs installs virtualenv, skip if already have it virtual env in the folder ‘venv‘ the virtual env compiled dependencies test dependencies If you are on OSX and some of the compiled dependencies fails to compile, try explicitly setting the arch flags and try again: $ export ARCHFLAGS="-arch i386 -arch x86_64" $ pip install -r requirements/compiled.txt If you are on Linux, you will need at least the following packages or their equivalent for your distro: libmysqlclient-dev libxslt-dev Now configure the application to run locally by creating your local settings file: $ cp bedrock/settings/local.py-dist bedrock/settings/local.py You shouldn’t need to customize anything in there yet. Sync the database and all of the external data locally. This gets product-details, security-advisories, credits, release notes, etc: $ bin/sync_all Lastly, you need to install node and the less package. Soon you won’t need this for local development but currently it compiles the LESS CSS code on the server-side: $ npm -g install less You don’t have to use npm to install less; feel free to install it however you want. Add the path to the LESS compiler (found by using which lessc) to bedrock/settings/local.py with the following line: LESS_BIN = ’/path/to/lessc’ 1.2.2 Run the tests Now that we have everything installed, let’s make sure all of our tests pass. This will be important during development so that you can easily know when you’ve broken something with a change. You should still have your virtualenv activated, so running the tests is as simple as: $ ./manage.py test Note: If your local tests run fine, but when you submit a pull-request our Jenkins (continuous integration service) instance tells you the tests failed, it could be due to the difference in settings between what you have in 4 Chapter 1. Contents playdoh Documentation, Release 1.0 settings/local.py and what Jenkins uses: settings/jenkins.py. You can run tests as close to Jenkins as possible by doing the following: $ JENKINS_HOME=1 ./manage.py test This tells Bedrock to use the jenkins settings. This will require you to have a local MySQL database server running and configured correctly, but may help you debug. Alternately you can move your settings/local.py to a backup, copy settings/jenkins.py to settings/local.py and tweak the DB settings yourself to make it work. Note: Another possible culprit when your tests pass but Jenkins doesn’t is that we use Python version 2.6 on our servers. We are hoping to upgrade to 2.7 in the future, but so far we’re using the default version that ships with RHEL (Red Hat Enterprise Linux) 6. If you use Python 2.7 specific features the tests will fail. 1.2.3 Make it run To make the server run, make sure you are inside a virtualenv, and then run the server: $ ./manage.py runserver If you are not inside a virtualenv, you can activate it by doing: $ source venv/bin/activate If you get the error “NoneType is not iterable”, you didn’t check out the latest product-details. See the above section for that. 1.2.4 Run it with the whole site If you need to run the whole site locally, you’ll need to first set up the PHP side, and then also set up to serve Bedrock from the same Apache server at /b/. That’s because the rewrite rules in the PHP and Apache config assume they can serve requests from Bedrock by rewriting them internally to have a /b/ on the front of their URLs. Important: Before continuing, go get the PHP side working. Then come back here. One way to add Bedrock to your local site, once you have the PHP side working, is to use runserver to serve Bedrock at port 8000 as above, then proxy to it from Apache. The whole virtual server config might end up looking like this: <VirtualHost *:80> ServerName mozilla.local VirtualDocumentRoot "/path/to/mozilla.com" RewriteEngine On RewriteOptions Inherit ProxyPass /b http://localhost:8000 ProxyPassReverse /b http://localhost:8000 ProxyPass /media http://localhost:8000/media ProxyPassReverse /media http://localhost:8000/media Include /path/to/bedrock/etc/httpd/global.conf </VirtualHost> But you might have better success using a real WSGI setup that is closer to what the real servers use. The following configuration is simplified from what the bedrock staging server uses. Assumptions: 1.2. Installing Bedrock 5 playdoh Documentation, Release 1.0 • A Red Hat or Debian-based Linux distribution. (Other distributions might not have Apache HTTP Server installed and configured the same way.) • Apache HTTP Server with php and mod_wsgi • Subversion mozilla.com checkout at /path/to/mozilla/mozilla.com • Subversion mozilla.org checkout at /path/to/mozilla/mozilla.com/org (ideally as an SVN external) • Bedrock checkout at /path/to/mozilla/bedrock Create a local config files for mozilla.com and mozilla.org: $ cp /path/to/mozilla.com/includes/config.inc.php-dist /path/to/mozilla.com/includes/config.inc.php $ cp /path/to/mozilla.com/org/includes/config.inc.php-dist /path/to/mozilla.com/org/includes/config.i Edit /etc/hosts and add: 127.0.0.1 mozilla.local Apache config - create file /etc/apache2/sites-available/mozilla.com: # Main site at /, django-bedrock at /b <VirtualHost *:80 *:81> ServerName mozilla.local ServerAdmin [email protected] DocumentRoot "/path/to/mozilla/mozilla.com" AddType application/x-httpd-php .php .html DirectoryIndex index.php index.html RewriteEngine On <Directory "/path/to/mozilla.com"> Options MultiViews FollowSymLinks -Indexes AllowOverride All </Directory> RewriteMap org-urls-410 txt:/path/to/mozilla.com/org-urls-410.txt RewriteMap org-urls-301 txt:/path/to/mozilla.com/org-urls-301.txt # In the path below, update "python2.6" to whatever version of python2 is provided. WSGIDaemonProcess bedrock_local python-path=/path/to/bedrock:/path/to/venv-for-bedrock/lib/python WSGIProcessGroup bedrock_local WSGIScriptAlias /b /path/to/bedrock/wsgi/playdoh.wsgi process-group=bedrock_local application-gro Alias /media /path/to/bedrock/media <Directory /path/to/bedrock/media> AllowOverride FileInfo Indexes </Directory> Include /path/to/bedrock/etc/httpd/global.conf </VirtualHost> Then enable the new site, build the css and js files, and finally restart apache: sudo a2ensite mozilla.com sudo a2enmod expires headers actions python manage.py collectstatic sudo service apache2 restart 6 Chapter 1. Contents playdoh Documentation, Release 1.0 Troubleshooting If you get Django error pages reporting I/O errors for .css files, it’s because not all the .css files were compiled before starting Apache and Apache does not have write permissions in the media directories. Running python manage.py collectstatic should solve it. Remember to run that command again anytime the css or less files change. If you change Python files, either restart Apache or touch playdoh.wsgi, so that the WSGI processes will be restarted and start running the new code. If you’re working on the rewrite rules in bedrock/etc/httpd/*.conf, be sure to restart Apache after any change. Apache doesn’t re-read those files after it has started. 1.2.5 Localization If you want to install localizations, just check out the locale directory: git svn clone https://svn.mozilla.org/projects/mozilla.com/trunk/locales/ locale # or svn checkout https://svn.mozilla.org/projects/mozilla.com/trunk/locales/ locale You can use git or svn to checkout the repo. Make sure that it is named locale. If you already have it checked out as locales, just do: ln -s locales locale You can read more details about how to localize content here. 1.2.6 Waffle Waffle is used to configure behavior and/or features of select pages on bedrock. Currently, Waffle switches are used to enable/disable Optimizely on the following URLs (Waffle switch names follow in parentheses): • / (mozorg-home-optimizely) • /firefox/desktop/ (firefox-desktop-optimizely) • /firefox/firstrun/ (firefox-firstrun-optimizely) • /firefox/installer-help/ (firefox-installer-help-optimizely) • /firefox/new/ (firefox-new-optimizely) • /firefox/whatsnew/ (firefox-whatsnew-optimizely) • /plugincheck/ (plugincheck-optimizely) To work with/test these Waffle/Optimizely switches locally, you must add the switches to your local database. For example: ./manage.py switch firefox-new-optimizely on --create You then must set an Optimizely project code in settings/local.py: # Optimize.ly project code OPTIMIZELY_PROJECT_ID = 12345 1.2. Installing Bedrock 7 playdoh Documentation, Release 1.0 Note: You are not required to set up Waffle & Optimizely as detailed above. If not configured, Waffle will treat the switches as set to off. For quick reference, to toggle a Waffle switch: ./manage.py switch firefox-desktop-optimizely off And to list all Waffle switches: ./manage.py switch -l 1.2.7 Upgrading On May 15th, 2013 we upgraded to a newer version of Playdoh. This brought with it a lot of structural changes to the code. Here are the required steps to get up and running again with the latest code: # get the code git pull origin master # update the submodules git submodule update --init --recursive # move your local settings file mv settings/local.py bedrock/settings/local.py # remove old empty directories rm -rf apps rm -rf settings rm -rf vendor-local/src/django rm -rf vendor-local/src/tower rm -rf vendor-local/src/jingo-minify That should do it. If you’re not able to run the tests at that point (python manage.py test) then there are a couple more things to try. 1. If you have a line like from settings.base import * in your bedrock/settings/local.py file, remove it. 2. If you were setting a logging level in your bedrock/settings/local.py file, you may now need to explicitly need to import it (import logging). Otherwise please pop into our IRC channel (#www on irc.mozilla.org) and we’ll be happy to help. 1.2.8 Notes A shortcut for activating virtual envs in zsh is . venv/bin/activate. The dot is the same as source. There’s a project called virtualenvwrapper that provides a better interface for managing/activating virtual envs, so you can use that if you want. 1.3 Vagrant Installation The Vagrant installation will help you work on the Python bedrock codebase and the PHP legacy codebase with a minimum amount of effort (hopefully). 8 Chapter 1. Contents playdoh Documentation, Release 1.0 This entire process will take between 30-50 minutes. For most of this time you will not be doing anything, Vagrant will be automagically downloading and configuring. This is a good time to have a cup of tea and/or coffee, possibly walk the dog. 1.3.1 Preparing Your System 1. Install Vagrant. Vagrant is a manager of VMs for development. Based on a configuration file, Vagrant will create a Virtual Machine, downloading and configuring everything you need to have a local environment running. This installation is tested using version: v1.4.3 Visit Vagrant’s download page. Do not install via apt-get, the version (at the time of writing) installed in debian wheezy appears broken. 2. Install Virtualbox. You are required to have virtualbox installed. This installation is tested with version 4.2.18 and can be downloaded at the virtualbox download page. • For Debian based systems: ~$ sudo apt-get install virtualbox 3. Install git. The bedrock code is revisioned using git <http://git-scm.org>. • For Debian based systems: ~$ sudo apt-get install git For other Linux distributions or operating systems visit Git’s download page. 4. Install svn. The legacy php code is revisioned using SVN. • For Debian based systems: ~$ sudo apt-get install subversion For other Linux distributions or operating systems visit SVN’s download page. 1.3.2 Build The Environment 1. Directory Setup. Create a top level directory to hold both bedrock and the legacy file system. You could call this directory ‘bedrock-legacy’. The following steps take place under that directory. 2. Using Git Clone Bedrock Repository. 1.3. Vagrant Installation 9 playdoh Documentation, Release 1.0 Bedrock is hosted at http://github.com/mozilla/bedrock. Clone the repository locally: ~bedrock-legacy$ git clone --recursive http://github.com/mozilla/bedrock Note: Make sure you use --recursive when checking the repo out! If you didn’t, you can load all the submodules with git submodule update --init --recursive. 3. Using SVN Checkout The Locale Repository. (Optional) If you would like to see localized versions of the site you will need to checkout the locale directory to the root of the bedrock directory you just cloned. Clone the repository locally: ~$ cd bedrock ~bedrock-legacy/bedrock$ svn checkout https://svn.mozilla.org/projects/mozilla.com/trunk/loc Note: You can read more details about how to localize content here. 4. Using SVN Checkout Mozilla.com PHP Repository. Mozilla.com PHP is hosted on https://svn.mozilla.org/projects/mozilla.com/trunk. Clone the repository locally: ~bedrock-legacy$ svn co https://svn.mozilla.org/projects/mozilla.com/trunk mozilla.com Note: At this stage you should have two directories side-by-side. bedrock and mozilla.com. 1.3.3 Configure The Environment 1. Configure Bedrock. Configure Bedrock by creating and editing the local settings file: ~bedrock-legacy$ cp bedrock/bedrock/settings/local.py-dist bedrock/bedrock/settings/local.py Add this line below LESS_PREPROCESS: LESS_BIN = ’/usr/local/bin/lessc’ 2. Configure Mozilla PHP. Configure the legacy site by creating and editing the local settings file: cd mozilla.com/includes cp config.inc.php-dist config.inc.php Set the following values: $config[’server_name’] = ’mozilla.local’; $config[’file_root’] = ’/srv/legacy’; 3. Set A Host Name. 10 Chapter 1. Contents playdoh Documentation, Release 1.0 We need to set a host name that you will use to access vagrant from a web-browser. You will need to add the following to your hosts file (note you may need sudo permissions). 192.168.10.55 mozilla.local The hosts file can be found in the following directories. • For Debian & OS X based systems: /etc/hosts • For Windows based systems c:\windows\system32\drivers\etc\hosts 1.3.4 Start Your Machine 1. Fire up vagrant. Now you need to build the virtual machine where Mozilla will live. Change into the cloned git directory and run vagrant. Note you must run this command in the directory that contains the Vagrantfile. ~$ cd bedrock ~bedrock-legacy/bedrock$ vagrant up --provision Note: The first time you run vagrant a VM image will be downloaded and the guest machine will be configured. You will be downloading more than 300Mb for the linux image and a bunch of additional downloading and configuration is going to happen. The total install can take 20 minutes on a fast machine. A decent internet connection is recommended. Note: Often the initial installation will time out while compiling node. If this happens just run the following command to re-sume the install: ~bedrock-legacy/bedrock$ vagrant provision 2. Update Product Details Bedrock needs to grab some information about Mozilla products to run. This is a one time update. To run the update you need to SSH into your Vagrant install and run the update script. SSH into your vagrant install ~bedrock-legacy/bedrock$ vagrant ssh CD Into The Top Level Bedrock Directory: ~$ cd /vagrant/ Update Product Details: /vagrant$ python manage.py update_product_details Exit /vagrant$ exit 3. Confirm Everything Is Setup. 1.3. Vagrant Installation 11 playdoh Documentation, Release 1.0 Confirm both bedrock and the legacy PHP site are working by visiting these urls. If everything looks right you are good to go! http://mozilla.local The mozilla homepage loading from bedrock. http://mozilla.local/en-US/about/legal.html A legacy page loading from PHP Note: The first time you load a page the CSS may not load. This is likely due to the CSS not being compiled. Doing a refresh will solve this problem. 1.3.5 Working & Workflow At this stage you should have a fully functional dev environment. You can work on files in your regular manner and follow the normal git workflow. 1.3.6 Tips & Tricks 1. Connect to your vagrant machine. You can connect to your vagrant machine, when it’s running, using: bedrock-legacy/bedrock$ vagrant ssh 2. Starting & Stopping Vagrant. Start ~$ vagrant up --provision Stop (vagrant is memory intensive - so if you are not using it best to stop it): ~$ vagrant halt 1.3.7 Troubleshooting Find us on irc in #webprod 1.4 Installing and Learning About the PHP Site The previous version of mozilla.org was written in PHP. The PHP codebase still serves some of the mozilla.org pages because we haven’t migrated everything over. A request runs through the following stack: • If the page exists in Bedrock, serve from Bedrock • If the page exists in the PHP site, serve from PHP • Else, serve a 404 page 1.4.1 History The PHP site has a long history and as a result, is a little quirky. If you are looking to work on the site and/or set it up locally, this page will be helpful to you. 12 Chapter 1. Contents playdoh Documentation, Release 1.0 mozilla.org, mozilla.com, and thunderbird used to be completely separate sites with different PHP codebases. In 2011 these sites were merged into one site. The merge is purely for aesthetics though. In the PHP side of mozilla.org, a few different PHP codebases coexist beside each other, and a combination of Apache and PHP magic bind them all together (one site to rule them all, or something like that). 1.4.2 Installing Apache Whether you’re installing just mozilla.com or also mozilla.org, there’s some common configuration required for Apache. 1. Install PHP. On Ubuntu, you can use these commands: sudo apt-get install libapache2-mod-php5 2. Enable the required modules. On Ubuntu, this should get most of them: sudo a2enmod actions expires headers php5 proxy proxy_http rewrite status vhost_alias but if Apache fails to start with errors about unknown directives, that probably means some other module also needs to be enabled. Bedrock The whole site now assumes Bedrock is also available. Even after following the instructions below, parts of the site will not work until/unless you also have Bedrock running locally. Or you might see an old version of a page from PHP, because the newer version is in Bedrock but the old version wasn’t removed from PHP. mozilla.com If you want to just work on the mozilla.com codebase (currently served at mozilla.org/firefox), follow these steps. You will only get the product pages. See mozilla.org for instructions on installing the org side of the site. For more details on why several codebases run the site, see How a Request is Handled. Note: This assumes you are using Apache with Unix. Windows might have different steps, please contact us if you need help. 1. Install it with these commands: svn co https://svn.mozilla.org/projects/mozilla.com/trunk mozilla.com cd mozilla.com/includes cp config.inc.php-dist config.inc.php 2. Open /includes/config.inc.php and set the server_name to “mozilla.local” (or whatever you will use) and file_root to the site’s path on the filesystem. 3. Set up mozilla.local to resolve to localhost. This is different for each OS, but a quick way on Linux/OS X is to add an entry to /etc/hosts: 127.0.0.1 mozilla.local 1.4. Installing and Learning About the PHP Site 13 playdoh Documentation, Release 1.0 4. Configure Apache to allow the site to run with a Directory and VirtualHost directive: This could go in the main Apache configuration file, or on Ubuntu, you might put this in /etc/apache2/sites-available/mozilla.com.conf. <Directory /path/to/mozilla.com> Options Includes FollowSymLinks MultiViews Indexes AllowOverride All Order Deny,Allow Allow from all </Directory> <VirtualHost *:80> ServerName mozilla.local VirtualDocumentRoot "/path/to/mozilla.com" </VirtualHost> Make sure to replace ServerName and /path/to/ to the correct values. On Ubuntu, you would then enable the site with: sudo a2ensite mozilla.com 5. You might need to set the DocumentRoot to the site if you can’t load any CSS files. We are looking to fix this. DocumentRoot “/path/to/mozilla/mozilla.com” You shouldn’t need anything else in the site config for mozilla.com. The .htaccess file at the root of mozilla.com contains the rest of the required configuration. 6. Restart Apache. On Ubuntu: sudo service apache2 restart If you go to http://mozilla.local/ you should see a page for downloading Firefox. mozilla.org If you need to work on mozilla.org, you need to install it as well. The installation process is identical to mozilla.com, with a few tweaks. Note: htaccess files do not work on mozilla.org. If you need to add anything to htaccess files, you must commit them to the mozilla.com codebase. See the section below about the merge for more info. 1. Make sure you install it as a subdirectory underneath mozilla.com named org. cd mozilla.com svn co https://svn.mozilla.org/projects/mozilla.org/trunk org cd org/includes cp config.inc.php-dist config.inc.php 2. Open /org/includes/config.inc.php and set the server_name to “mozilla.local” (or whatever you will use) and file_root to the site’s path on the filesystem (including the org subdirectory). 3. In addition, set the ‘js_prefix’, ‘img_prefix’, ‘style_prefix’ config values to ‘/org’. That is necessary. 4. If you need the archive redirects to work, you need to add the RewriteMap directives to your Apache config for the site. Inside the VirtualHost section that you made while installing mozilla.com, add this: RewriteMap org-urls-410 txt:/path/to/mozilla.com/org-urls-410.txt RewriteMap org-urls-301 txt:/path/to/mozilla.com/org-urls-301.txt 14 Chapter 1. Contents playdoh Documentation, Release 1.0 5. Depending on your system settings, you might see warnings about relying on the system’s timezone settings. If you get this, add the following to the config.inc.php for mozilla.org: date_default_timezone_set(’America/New_York’); You can look up the correct timezone here. That should be it. If you go to http://mozilla.local/ (or whatever local server you set it to) you should see the org home page. Thunderbird The thunderbird site has been completely merged in with mozilla.org, so you can install it by installing mozilla.org. It will be served at /thunderbird. 1.4.3 Workflow If you are working on a bug, please follow these steps: 1. Commit your work to trunk 2. Comment on the bug and add the revision in the whiteboard field in the form “r=10000”. Multiple revisions should be comma-delimited, like “r=10000,10001”. You can add the revision in the comment too if you want people to have a link to the changes. 3. Add the keyword “qawanted” when finished 4. When all the work is done and has been QAed, mark as resolved. We release a batch of resolved bugs every Tuesday. Other bugs can go out between releases, but by default resolved bugs tagged with the current milestone will go out the next Tuesday. Stage isn’t used for much, but it’s useful for times when we are very careful about rolling out something. You typically don’t need to worry about it. When bugs are pushed live, they are pushed to stage and production at the same time. Rolling out code So you want to rollout a bug into production? If you look at our workflow, there should be some SVN revisions logged into the whiteboard of the bug. If not, you need to track down which revisions to push from the comments. Once you have this list, you need to merge them to the branches tags/stage and tags/production. If the revisions are already pushed to stage, only do the latter. These are the commands: cd tags/stage svn merge --ignore-ancestry -c<revs> ../../trunk svn commit -m ’merged <rev> from trunk for bug <id>’ <revs> is a single rev or comma-delimited like “10000,10001,10002”. Do the same for tags/production. Always format the log message like the above. You must use –ignore-ancestry also to avoid bad things. We wrote a script to automate this if you are doing this a lot. You can find it it on trunk in /bin/rollout. The usage looks like this: Usage: rollout <bug-id> <revs> <branch> <revs> and <branch> are optional 1.4. Installing and Learning About the PHP Site 15 playdoh Documentation, Release 1.0 $ cd mozilla.com $ rollout 654321 # must have trunk, tags/stage, and tags/production checked out here Merging into tags/stage... --- Merging r654321 into ’.’: <svn output> Continue? y/n [n]y Committing tags/stage... Merging into tags/production... --- Merging r654321 into ’.’: <svn output> Continue? y/n [n]y Committing tags/production... The script parses the revisions and branch from the whiteboard data in bugzilla, and merges it from trunk to stage and production. If the branch is already stage (b=stage in the whiteboard) it just merges it to production. After it does the merges, it asks you if you want to continue. If you saw conflicts, you shouldn’t continue and you should fix the conflicts and either finish the rollout by hand or update the bugzilla whiteboard and run the command again. 1.4.4 How a Request is Handled Magic should always be documented, so let’s look at exactly how all the PHP sites work together to handle a mozilla.org request. mozilla.org is made up of three sites: • mozilla.com (the product pages) • mozilla.org (mofo) • mozillamessaging.com (thunderbird) These three sites are now all merged into http://mozilla.org/. However, on the server a request can be handled by three different codebases. We’ll refer to the mozilla.com codebase as moco, mozilla.org codebase as mofo, and messaging as thunderbird. moco is the primary codebase. A request goes through the following steps: • If the URL exists in the mofo codebase, load the page from there • If the URL exists in the thunderbird codebase, load from there • Else, let moco handle the URL like normal The merge magic is installed into moco’s htaccess and PHP files. We let moco become the primary codebase because if there’s any error in the merge code, we can’t afford to break the main Firefox product pages. There’s also more developer attention on moco. Special Note: Only mozilla.com’s .htaccess files are processed by Apache. All the others have been merged in so you shouldn’t add anything to them. Please add all htaccess rules inthe mozilla.com codebase. 16 Chapter 1. Contents playdoh Documentation, Release 1.0 Merge Magic How we implement the merge is really important. Performance, site breakage, and amount of work to move things around are all serious considerations. The merge is meant to be temporary as the site is moving to Python, so it’s not worth the effort to literally merge all the PHP code together. It’s also important to still allow the mofo and moco codebases to be run individually. We don’t want to suddenly break it for people who have it locally checked out (short-term wise). Finally, the code of each site also dictated possible solutions. There’s a lot of edge cases in each site so need to make sure we don’t break anything. Here’s how the merge magic was implemented: Short version: • Check out the mofo codebase under moco as the subdirectory org. • Redirect all mofo URLs to a PHP handler which loads those pages, do the same for thunderbird • Fix loading of images, css, and js by setting prefix config values and more rewrites • Merge .htaccess files into the moco codebase Long version: • Check out the mofo codebase under moco as the subdirectory org. – Thunderbird is a folder under org, at /org/thunderbird • Generate a list of top-level folders in the org site and use Apache rewrites to redirect all those URLs to a special php handler • Write the special php handler to load mofo pages. This is basically a port of mofo’s prefetch.php • Write a similar handler for the thunderbird pages and redirect all /thunderbird URLs to it • Fix loading of assets – Set config values to load assets with the “/org” prefix – For bad code that doesn’t use the config, use apache rewrites to redirect images and script to the respective folder in “/org”. These two folders don’t conflict with the moco codebase. The style directory conflicts, so make sure all code uses the config prefix value. – Redirect any other asset directory to use the “/org” prefix (/thunderbird/img/, etc) • Merge .htacess files – The biggest side effect of this is that only moco htaccess files are processed, but we should consolidate things anyway – Move the redirects and other appropriate rules from mofo’s htaccess to moco’s – Optimize the crazy amount of 301 and 410 redirects from mofo, mostly archive redirects, using RewriteMap – Test to make sure everything’s working, implement special rewrites or org-handler.php hacks to fix any breakage • Check file extensions for any leftover static types and rewrite them to be served by Apache The final result is the moco codebase which dispatches a lot of URLs to the mofo and thunderbird codebases. 1.4. Installing and Learning About the PHP Site 17 playdoh Documentation, Release 1.0 1.5 Localization The site is fully localizable. Localization files are not shipped with the code distribution, but are available on SVN: $ git svn clone https://svn.mozilla.org/projects/mozilla.com/trunk/locales/ locale # or $ svn checkout https://svn.mozilla.org/projects/mozilla.com/trunk/locales/ locale 1.5.1 .lang files Bedrock supports a workflow similar to gettext. You extract all the strings from the codebase, then merge them into each locale to get them translated. The files containing the strings are called ”.lang files” and end with a .lang extension. To extract all the strings from the codebase, run: $ ./manage.py l10n_extract If you’d only like to extract strings from certain files, you may optionally list them on the command line: $ ./manage.py l10n_extract apps/mozorg/templates/mozorg/contribute.html Command line glob matching will work as well if you want all of the html files in a directory for example: $ ./manage.py l10n_extract apps/mozorg/templates/mozorg/*.html That will use gettext to get all the needed localizations from python and html files, and will convert the result into a bunch of .lang files inside locale/templates. This directory represents the “reference” set of strings to be translated, and you are free to modify or split up .lang files here as needed (just make sure they are being referenced correctly, from the code, see Which .lang file should it use?). To merge new strings into locale directories, run: $ ./manage.py l10n_merge If you want to merge only specific locales, you can pass any number of them as arguments: $ ./manage.py l10n_merge fr de Translating with .lang files To translate a string from a .lang file, simply use the gettext interface. In a jinja2 template: <div>{{ _(’Hello, how are you?’) }}<div> <div>{{ _(’<a href="%s">Click here</a>’)|format(’http://mozilla.org/’) }}</div> <div>{{ _(’<a href="%(url)s">Click here</a>’)|format(url=’http://mozilla.org/’) }}</div> Note the usage of variable substitution in the latter examples. It is important not to hardcode URLs or other parameters in the string. jinja’s format filter lets us apply variables outsite of the string. You can provide a one-line comment to the translators like this: 18 Chapter 1. Contents playdoh Documentation, Release 1.0 {# L10n: "like" as in "similar to", not "is fond of" #} {{ _(’Like this:’) }} The comment will be included in the .lang files above the string to be translated. In a Python file, use lib.l10n_utils.dotlang._ or lib.l10n_utils.dotlang._lazy, like this: from lib.l10n_utils.dotlang import _lazy as _ sometext = _(’Foo about bar.’) You can provide a one-line comment to the translators like this: # L10n: "like" as in "similar to", not "is fond of" sometext = _(’Like this:’) The comment will be included in the .lang files above the string to be translated. There’s another way to translate content within jinja2 templates. If you need a big chunk of content translated, you can put it all inside a trans block. {% trans %} <div>Hello, how are you</div> {% endtrans %} {% trans url=’http://mozilla.org’ %} <div><a href="{{ url }}">Click here</a></div> {% endtrans %} Note that it also allows variable substitution by passing variables into the block and using template variables to apply them. A general good practice is to enclose short strings in l10n calls (trans blocks or gettext wrapper). If you have a paragraph with several sentences, it is better to wrap each sentence in its own call than the whole paragraph. That makes it more digestable for localizers and avoids having a whole paragraph invalidated for a change to one sentence only. Example: <p> {{_(’As a result, more countries and mobile phone operators will be selling Firefox in the future.’)} {{_(’Our operator partners will distribute the phones through a variety of locally-specific channels. </p> Which .lang file should it use? Translated strings are split across several .lang files to make it easier to manage separate projects and pages. So how does the system know which one to use when translating a particular string? • All translations from Python files are put into main.lang. This should be a very limited set of strings and most likely should be available to all pages. • Templates always load in main.lang, download_button.lang, and newsletter.lang • Additionally, each template has its own .lang file, so a template at mozorg/firefox.html would use the .lang file at <locale>/mozorg/firefox.lang. • Templates can override which lang files are loaded. The above 3 global ones are always loaded, but instead of loading <locale>/mozorg/firefox.lang, the template can specify a list of additional lang files to load with a template block: 1.5. Localization 19 playdoh Documentation, Release 1.0 {% add_lang_files "foo" "bar" %} That will make the page load foo.lang and bar.lang in addition to main.lang, download_button.lang, and newsletter.lang. When strings are extracted from a template, that are added to the template-specific .lang file. If the template explicitly specifies .lang files like above, it will add the strings to the first .lang file specified, so extracted strings from the above template would go into foo.lang. You can similarly specify extra lang files in your Python source as well. Simply add a module-level constant in the file named LANG_FILES. The value should be either a string, or a list of strings, similar to the add_lang_files tag above. # forms.py from lib.l10n_utils.dotlang import _ LANG_FILES = [’foo’, ’bar’] sometext = _(’Foo about bar.’) This file’s strings would be extracted to foo.lang, and the lang files foo.lang, bar.lang, main.lang, download_button.lang, and newsletter.lang would be searched for matches in that order. 1.5.2 l10n blocks Bedrock also has a block-based translation system that works like the {% block %} template tag, and marks large sections of translatable content. This should not be used very often; lang files are the preferred way to translate content. However, there may be times when you want to control a large section of a page and customize it without caring very much about future updates to the English page. A Localizers’ guide to l10n blocks Let’s look at how we would translate an example file from English to German. The English source template, created by a developer, lives under apps/appname/templates/appname/example.html and looks like this: {% extends "base.html" %} {% block content %} <img src="someimage.jpg"> {% l10n foo, 20110801 %} <h1>Hello world!</h1> {% endl10n %} <hr> {% l10n bar, 20110801 %} <p>This is an example!</p> {% endl10n %} {% endblock %} The l10n blocks mark content that should be localized. Realistically, the content in these blocks would be much larger. For a short string like above, please use lang files. We’ll use this trivial code for our example though. 20 Chapter 1. Contents playdoh Documentation, Release 1.0 The l10n blocks are named and tagged with a date (in ISO format). The date indicates the time that this content was updated and needs to be translated. If you are changing trivial things, you shouldn’t update it. The point of l10n blocks is that localizers completely customize the content, so they don’t care about small updates. However, you may add something important that needs to be added in the localized blocks; hence, you should update the date in that case. When the command ./manage.py l10n_extract is run, it generates the corresponding files in the locale folder (see below for more info on this command). The german version of this template is created at locale/de/templates/appname/example.html. The contents of it are: {% extends "appname/example.html" %} {% l10n foo %} <h1>Hello world!</h1> {% endl10n %} {% l10n bar %} <p>This is an example!</p> {% endl10n %} This file is an actual template for the site. It extends the main template and contains a list of l10n blocks which override the content on the page. The localizer just needs to translate the content in the l10n blocks. When the reference template is updated with new content and the date is updated on an l10n block, the generated l10n file will simply add the new content. It will look like this: {% extends "appname/example.html" %} {% l10n foo %} <h1>This is an English string that needs translating.</h1> {% was %} <h1>Dies ist ein English string wurde nicht.</h1> {% endl10n %} {% l10n bar %} <p>This is an example!</p> {% endl10n %} Note the was block in foo. The old translated content is in there, and the new content is above it. The was content is always shown on the site, so the old translation still shows up. The localizer needs to update the translated content and remove the was block. Generating the locale files $ ./manage.py l10n_check This command will check which blocks need to be translated and update the locale templates with needed translations. It will copy the English blocks into the locale files if a translation is needed. You can specify a list of locales to update: $ ./manage.py l10n_check fr $ ./manage.py l10n_check fr de es 1.5. Localization 21 playdoh Documentation, Release 1.0 1.5.3 Currency When dealing with currency, make a separate gettext wrapper, placing the amount inside a variable. You should also include a comment describing the intent. For example: {# L10n: Inserts a sum in US dollars, e.g. ’$100’. Adapt the string in your translation for your loca {{ _(’$%(sum)s’)|format(sum=’15’) }} 1.5.4 CSS If a localized page needs some locale-specific style tweaks, you can add the style rules to the page’s stylesheet like this: html[lang=”it”] { #features li { font-size: 20px; } } html[dir=”rtl”] { #features { float: right; } } If a locale needs site-wide style tweaks, font settings in particular, you can add the rules to /media/css/l10n/{{LANG}}/intl.css. Pages on Bedrock automatically includes the CSS in the base templates with the l10n_css helper function. The CSS may also be loaded directly from other Mozilla sites with such a URL: //mozorg.cdn.mozilla.net/media/css/l10n/{{LANG}}/intl.css. Open Sans, the default font on mozilla.org, doesn’t offer non-Latin glyphs. intl.css can have @font-face rules to define locale-specific fonts using custom font families as below: • X-LocaleSpecific-Light: Used in combination with Open Sans Light. The font can come in 2 weights: normal and optionally bold • X-LocaleSpecific: Used in combination with Open Sans Regular. The font can come in 2 weights: normal and optionally bold • X-LocaleSpecific-Extrabold: Used in combination with Open Sans Extrabold. The font weight is 800 only Here’s an example of intl.css: @font-face { font-family: X-LocaleSpecific-Light; font-weight: normal; src: local(mplus-2p-light), local(Meiryo); } @font-face { font-family: X-LocaleSpecific-Light; font-weight: bold; src: local(mplus-2p-medium), local(Meiryo-Bold); } @font-face { font-family: X-LocaleSpecific; font-weight: normal; src: local(mplus-2p-regular), local(Meiryo); } @font-face { font-family: local(Meiryo-Bold); 22 X-LocaleSpecific; font-weight: bold; src: local(mplus-2p-bold), Chapter 1. Contents playdoh Documentation, Release 1.0 } @font-face { font-family: X-LocaleSpecific-Extrabold; font-weight: 800; src: local(mplus-2p-black), local(Meiryo-Bold); } Localizers can specify locale-specific fonts in one of the following ways: • Choose best-looking fonts widely used on major platforms, and specify those with the src: local(name) syntax * Find a best-looking free Web font, add the font files to /media/fonts/, and specify those with the src: url(path) syntax * Create a custom Web font to complement missing glyphs in Open Sans, add the font files to /media/fonts/, and specify those with the src: url(path) syntax. The M+ font family offers various international glyphs and looks similar to Open Sans. You can create a subset of the M+ 2c font using a tool found on the Web. See Bug 776967 for the Fulah (ff) locale’s example. Developers should use the .open-sans mixin instead of font-family: ’Open Sans’ to specify the default font family in CSS. This mixin has both Open Sans and X-LocaleSpecific so locale-specific fonts, if defined, will be applied to localized pages. The variant mixins, .open-sans-light and .open-sans-extrabold, are also available. 1.6 Developing on Bedrock 1.6.1 Writing URL Patterns URL patterns should be as strict as possible. It should begin with a ^ and end with /$ to make sure it only matches what you specifiy. It also forces a trailing slash. You should also give the URL a name so that other pages can reference it instead of hardcoding the URL. Example: url(r’^channel/$’, channel, name=’mozorg.channel’) Bedrock comes with a handy shortcut to automate all of this: from bedrock.mozorg.util import page page(’channel’, ’mozorg/channel.html’) You don’t even need to create a view. It will serve up the specified template at the given URL (the first parameter). You can also pass template data as keyword arguments: page(‘channel’, ‘mozorg/channel.html’, latest_version=product_details.firefox_versions[’LATEST_FIREFOX_VERSION’]) The variable latest_version will be available in the template. 1.6.2 Embedding images Images should be included on pages using helper functions. static() For a simple image, the static() function is used to generate the image URL. For example: <img src="{{ static(’img/firefox/new/firefox-logo.png’) }}" alt="Firefox" /> will output an image: 1.6. Developing on Bedrock 23 playdoh Documentation, Release 1.0 <img src="/media/img/firefox/new/firefox-logo.png" alt="Firefox"> high_res_img() For images that include a high-resolution alternative for displays with a high pixel density, use the high_res_img() function: high_res_img(’img/firefox/new/firefox-logo.png’, {’alt’: ’Firefox’, ’width’: ’200’, ’height’: ’100’}) The high_res_img() function will automatically look for the image in the URL parameter suffixed with ‘-high-res’, e.g. img/firefox/new/firefox-logo-high-res.png and switch to it if the display has high pixel density. high_res_img() supports localized images by setting the ‘l10n’ parameter to True: high_res_img(’img/firefox/new/firefox-logo.png’, {’l10n’: True, ’alt’: ’Firefox’, ’width’: ’200’, ’he When using localization, high_res_img() will look for images in the appropriate locale folder. In the above example, for the de locale, both standard and high-res versions of the image should be located at media/img/l10n/de/firefox/new/. l10n_img() Images that have translatable text can be handled with l10n_img(): <img src="{{ l10n_img(’firefox/os/have-it-all/messages.jpg’) }}" /> The images referenced by l10n_img() must exist in media/img/l10n/, so for above example, the images could include media/img/l10n/en-US/firefox/os/have-it-all/messages.jpg and media/img/l10n/es-ES/firefox/os/have-itall/messages.jpg. platform_img() Finally, for outputting an image that differs depending on the platform being used, the platform_img() function will automatically display the image for the user’s browser: platform_img(’img/firefox/new/browser.png’, {’alt’: ’Firefox screenshot’}) platform_img() will automatically look for the images browser-mac.png, browser-win.png, browser-linux.png, etc. Platform image also supports hi-res images by adding ‘high-res’: True to the list of optional attributes. platform_img() supports localized images by setting the ‘l10n’ parameter to True: platform_img(’img/firefox/new/firefox-logo.png’, {’l10n’: True, ’alt’: ’Firefox screenshot’}) When using localization, platform_img() will look for images in the appropriate locale folder. In the above example, for the es-ES locale, all platform versions of the image should be located at media/img/l10n/es-ES/firefox/new/. 1.6.3 Writing Views You should rarely need to write a view for mozilla.org. Most pages are static and you should use the page expression documented above. If you need to write a view and the page has a newsletter signup form in the footer (most do), make sure to handle this in your view. Bedrock comes with a function for doing this automatically: 24 Chapter 1. Contents playdoh Documentation, Release 1.0 from bedrock.mozorg.util import handle_newsletter from django.views.decorators.csrf import csrf_exempt @csrf_exempt def view(request): ctx = handle_newsletter(request) return l10n_utils.render(request, ’app/template.html’, ctx) You’ll notice a few other things in there. You should use the l10n_utils.render function to render templates because it handles special l10n work for us. Since we’re handling the newsletter form post, you also need the csrf_exempt decorator. Make sure to namespace your templates by putting them in a directory named after your app, so instead of templates/template.html they would be in templates/blog/template.html if blog was the name of your app. 1.6.4 Coding Style Guides • Mozilla Python Style Guide • Mozilla HTML Style Guide • Mozilla JS Style Guide • Mozilla CSS Style Guide Use the .open-sans, .open-sans-light and .open-sans-extrabold mixins to specify font families to allow using international fonts. See the :ref: CSS<l10n> section in the l10n doc for details. Use the .font-size() mixin to generate root-relative font sizes. You can declare a font size in pixels and the mixin will convert it to an equivalent rem (root em) unit while also including the pixel value as a fallback for older browsers that don’t support rem. This is preferable to declaring font sizes in either fixed units (px, pt, etc) or element-relative units (em, %). See this post by Jonathan Snook for more info. 1.6.5 Configuring your code editor Bedrock includes an .editorconfig file in the root directory that you can use with your code editor to help maintain consistent coding styles. Please see editorconfig.org. for a list of supported editors and available plugins. 1.7 JavaScript Libraries • mozilla-pager.js • mozilla-accordion.js 1.8 How to contribute Before diving into code it might be worth reading through the Developing on Bedrock documentation, which contains useful information and links to our coding guidelines for Python, Django, JavaScript and CSS. 1.7. JavaScript Libraries 25 playdoh Documentation, Release 1.0 1.8.1 Git workflow When you want to start contributing, you should create a branch from master. This allows you to work on different project at the same time: git checkout master git checkout -b topic-branch To keep your branch up-to-date, assuming the mozilla repository is the remote called mozilla: git git git git git fetch mozilla checkout master merge mozilla/master checkout topic-branch rebase master If you need more Git expertise, a good resource is the Git book. Once you’re done with your changes, you’ll need to describe those changes in the commit message. 1.8.2 Git commit messages Commit messages are important when you need to understand why something was done. • First, learn how to write good git commit messages. • All commit messages must include a bug number. You can put the bug number on any line, not only the first one. • If you use the syntax bug xxx, Github will reference the commit into Bugzilla. With fix bug xxx, it will even close the bug once it goes into master. If you’re asked to change your commit message, you can use these commands: git commit --amend # -f is doing a force push because you modified the history git push -f my-remote topic-branch 1.8.3 Submitting your work In general, you should submit your work with a pull request to master. If you are working with other people or you want to put your work on a demo server, then you should be working on a common topic branch. Once your code has been positively reviewed, it will be deployed shortly after. So if you want feedback on your code but it’s not ready to be deployed, you should note it in the pull request. 1.8.4 Squashing your commits Should your pull request contain more than one commit, sometimes we may ask you to squash them into a single commit before merging. You can do this with git rebase. As an example, let’s say your pull request contains two commits. To squash them into a single commit, you can follow these instructions: git rebase -i HEAD~2 26 Chapter 1. Contents playdoh Documentation, Release 1.0 You will then get an editor with your two commits listed. Change the second commit from pick to fixup, then save and close. You should then be able to verify that you only have one commit now with git log. To push to GitHub again, because you “altered the history” of the repo by merging the two commits into one, you’ll have to git push -f instead of just git push. 1.8.5 Getting a new Bedrock page online On our servers, Bedrock pages are accessible behind the /b/ prefix. So if a page is accessible at this URL locally: http://localhost:8000/foo/bar then on our servers, it will be accessible at: http://www.mozilla.org/b/foo/bar When you’re ready to make a page available to everyone, we need to remove that /b/ prefix. We handle that with Apache RewriteRule. Apache config files that are included into the server’s config are in the bedrock code base in the etc/httpd directory. In there you’ll find a file for each of the environments. You’ll almost always want to use global.conf unless you have a great reason for only wanting the config to stay on one of the non-production environments. In that file you’ll add a RewriteRule that looks like the following: # bug 123456 RewriteRule ^/(\w{2,3}(?:-\w{2}(?:-mac)?)?/)?foo/bar(/?)$ /b/$1foo/bar$2 [PT] This is a lot simpler than it looks. The first large capture is just what’s necessary to catch every possible locale code. After that it’s just your new path. Always capture the trailing slash as we want that to hit django so it will redirect. Note: It’s best if the RewriteRule required for a new page is in the original pull request. This allows it to flow through the push process with the code and for it to go live as soon as it’s on the production server. It’s also one less review and pull-request for us to manage. 1.8.6 Server architecture Demos • URLs: http://www-demo1.allizom.org/ , http://www-demo2.allizom.org/ and http://www-demo3.allizom.org/ • PHP SVN branch: trunk, updated every 10 minutes • Bedrock locale SVN branch: trunk, updated every 10 minutes • Bedrock Git branch: any branch we want, manually updated Dev • URL: http://www-dev.allizom.org/ • PHP SVN branch: trunk, updated every 10 minutes • Bedrock locale SVN branch: trunk, updated every 10 minutes • Bedrock Git branch: master, updated every 10 minutes Stage • URL: http://www.allizom.org/ 1.8. How to contribute 27 playdoh Documentation, Release 1.0 • PHP SVN branch: tags/stage, updated every 10 minutes • Bedrock locale SVN branch: trunk, updated every 10 minutes • Bedrock Git branch: master, updated manually Production • URL: http://www.mozilla.org/ • PHP SVN branch: tags/production, updated every 10 minutes • Bedrock locale SVN branch: trunk, updated every 10 minutes • Bedrock Git branch: master, updated manually We use Chief for the manual deploys. https://www.mozilla.org/media/revision.txt. You can check the currently deployed git commit by checking If you want to know more and you have an LDAP account, you can check the IT documentation. 1.8.7 Pushing to production We’re doing pushes as soon as new work is ready to go out. After doing a push, the “pusher” needs to update the bugs that have been pushed with a quick message stating that the code was deployed. Chief will send on #www a URL with all commits that have been deployed. If you’d like to see the commits that will be deployed before the push run the following command: ./bin/open-compare.py This will discover the currently deployed git hash, and open a compare URL at github to the latest master. Look at open-compare.py -h for more options. 1.9 Using Grunt 1.9.1 Introduction If you haven’t used Grunt before, be sure to check out the Getting Started guide, as it explains how to create a Gruntfile as well as install and use Grunt plugins. Bedrock provides a Gruntfile.js in the root of the project to make local development easier, by automating common tasks such as: • Compiling CSS when a LESS file changes. • Running JSHint when a JavaScript file changes. • Live reloading in the browser whenever an HTML, CSS of JavaScript file changes. 1.9.2 Installation Grunt and Grunt plugins are installed and managed via npm, the Node package manager. In order to get started, you’ll want to install Grunt’s command line interface (CLI) globally. You may need to use sudo (for OSX, *nix, BSD etc) or run your command shell as Administrator (for Windows) to do this: 28 Chapter 1. Contents playdoh Documentation, Release 1.0 npm install -g grunt-cli You may also want to install JSHint globally using: npm install -g jshint Finally, install the dependencies that the bedrock Gruntfile needs: npm install 1.9.3 Usage To start the grunt task runner, simply run: grunt To enable live-reload in the browser bedrock/settings/local.py: you must set USE_GRUNT_LIVERELOAD to True in USE_GRUNT_LIVERELOAD = True In the root directory you will also find a .jshintrc-dist file which contains a basic set of defaults for running JSHint. If you wish to use these defaults with Grunt then copy the contents to a local .jshintrc file: cp .jshintrc-dist .jshintrc 1.9.4 Testing Bedrock has a suite of JavaScript unit tests written using Jasmine and Sinon. You can run these tests on the command line using Karma test runner and Firefox. To perform a single run of the test suite, type the following command: grunt test Note: The Tabzilla tests require that you have your local bedrock development server running on port 8000. 1.9.5 Cleaning generated CSS files Bedrock uses Less to generate CSS files. Sometimes during development you may want to clear out your cached CSS that gets generated. To make this easier, you can clear all *.less.css files located in media/css/ directories with the following command: grunt clean 1.10 Newsletters Bedrock includes support for signing up for and managing subscriptions and preferences for Mozilla newsletters. By default, every page’s footer has a form to signup for the default newsletter, “Firefox & You”. 1.10. Newsletters 29 playdoh Documentation, Release 1.0 1.10.1 Features • ability to subscribe to a newsletter from a page’s footer area. Many pages on the site might include this. • whole pages devoted to subscribing to one newsletter, often with custom text, branding, and layout • newsletter preference center - allow user to change their email address, preferences (e.g. language, HTML vs. text), which newsletters they’re subscribed to, etc. Access is limited by requiring a user-specific token in the URL (it’s a UUID). The full URL is included as a link in each newsletter sent to the user, which is the only way (currently) they can get the token. • landing pages that user ends up on after subscribing. These can vary depending on where they’re coming from. 1.10.2 Newsletters Newsletters have a variety of characteristics. Some of these are implemented in Bedrock, others are transparent to Bedrock but implemented in the basket back-end that provides our interface to the newsletter vendor. • Public name - the name that is displayed to users, e.g. “Firefox Weekly Tips”. • Internal name- a short string that is used internal to Bedrock and basket to identify a newsletter. Typically these are lowercase strings of words joined by hyphens, e.g. “firefox-tips”. This is what we send to basket to identify a newsletter, e.g. to subscribe a user to it. • Show publicly - pages like the newsletter preferences center show a list of unsubscribed newsletters and allow subscribing to them. Some newsletters aren’t included in that list by default (though they are shown if the user is already subscribed, to let them unsubscribe). • Languages - newsletters are available in a particular set of languages. Typically when subscribing to a newsletter, a user can choose their preferred language. We should try not to let them subscribe to a newsletter in a language that it doesn’t support. The backend only stores one language for the user though, so whenever the user submits one of our forms, whatever language they last submitted is what is saved for their preference for everything. • Welcome message - each newsletter can have a canned welcome message that is sent to a user when they subscribe to it. Newsletters should have both an HTML and a text version of this. • Drip campaigns - some newsletters implement so-called drip campaigns, in which a series of canned messages are dribbled out to the user over a period of time. E.g. 1 week after subscribing, they might get message 1; a week later, message 2, and so on until all the canned messages have been sent. Because drip campaigns depend on the signup date of the user, we’re careful not to accidentally change the signup date, which could happen if we sent redundant subscription commands to our backend. 1.10.3 Bedrock and Basket Bedrock is the user-facing web application. It presents an interface for users to subscribe and manage their subscriptions and preferences. It does not store any information. It gets all newsletter and user-related information, and makes updates, via web requests to the Basket server. The Basket server implements an HTTP API for the newsletters. The front-end (Bedrock) can make calls to it to retrieve or change users’ preferences and subscriptions, and information about the available newsletters. Basket implements some of that itself, and other functions by calling the newsletter vendor’s API. Details of that are outside the scope of this document, but it’s worth mentioning that both the user token (UUID) and the newsletter internal name mentioned above are used only between Bedrock and Basket. 30 Chapter 1. Contents playdoh Documentation, Release 1.0 1.10.4 URLs Here are a few important URLs implemented. These were established before Bedrock came along and so are unlikely to be changed. (Not all of these might be implemented in Bedrock yet.) /newsletter/ - subscribe to ‘mozilla-and-you’ newsletter (public name: “Firefox & You”) /newsletter/hacks.mozilla.org/ - subscribe to ‘app-dev’ newsletter (‘Firefox Apps & Hacks’). This one is displayed as a frame inside some other page(s), so it works differently than the other signup pages. /newsletter/existing/USERTOKEN/ - user management of their preferences and subscriptions 1.10.5 Configuration Currently, information about the available newsletters is configured in Basket. See Basket for more information. 1.10.6 Footer signup Customize the footer signup form by overriding the email_form template block. For example, to have no signup form: {% block email_form %}{% endblock %} The default is: {% block email_form %}{{ email_newsletter_form() }}{% endblock %} which gives a signup for Firefox & You. You can pass parameters to the macro email_newsletter_form to change that. For example, the newsletter_id parameter controls which newsletter is signed up for, and title can override the text: {% block email_form %} {{ email_newsletter_form(’app-dev’, _(’Sign up for more news about the Firefox Marketplace.’)) }}) {% endblock %} Pages can control whether country or language fields are included by passing include_language=[True|False] and/or include_country=[True|False]. You can also use the same form outside a page footer by passing footer=False to the macro. 1.10.7 Creating a signup page Start with a template that extends ’newsletter/one_newsletter_signup.html’. It’s probably simplest to copy an existing one, like ’newsletter/mobile.html’. Set the newsletter_title and newsletter_id variables and override at least the page_title and newsletter_content blocks: {% set newsletter_title = _(’Firefox and You’) %} {% set newsletter_id = ’mozilla-and-you’ %} {% block page_title %}{{ newsletter_title }}{% endblock %} {% block newsletter_content %} <div id="main-feature"> 1.10. Newsletters 31 playdoh Documentation, Release 1.0 <h2>Subscribe to <span>about:mobile</span>!</h2> <p>Our about:mobile newsletter brings you the latest and greatest news from the Mozilla contributor community. </p> </div> {% endblock %} Then add a url to newsletter/urls.py: # "about:mobile" page(’newsletter/about_mobile’, ’newsletter/mobile.html’), 1.11 Tabzilla Tabzilla is the universal tab displayed on Mozilla websites. Adding the universal tab to a site requires: 1. Add the static tab link (example below) to the top of your template: <a href="https://www.mozilla.org/" id="tabzilla">mozilla</a> 2. Include the tabzilla CSS by adding the following tag inside the <head> of your template: <link href="//mozorg.cdn.mozilla.net/media/css/tabzilla-min.css" rel="stylesheet" /> 3. Include the tabzilla.js file in your template (preferably just before the </body>): <script src="//mozorg.cdn.mozilla.net/tabzilla/tabzilla.js"></script> This will choose the best locale for your visitor. If you prefer to force the locale, you can use: <script src="//mozorg.cdn.mozilla.net/{locale}/tabzilla/tabzilla.js"></script> Where {locale} is the language in which you’d like Tabzilla to be loaded (e.g. fr or de). If Tabzilla is not yet translated into said locale the user will get the en-US version. Note: Tabzilla uses jQuery. If your site already includes jQuery be sure to place the Tabzilla script tag after the one for jQuery. Tabzilla will use the existing jQuery if available and a supported version, otherwise it will load its own version of jQuery. That the source file URLs begin with // is not a typo. This is a protocol-relative URL which allows the resource to be loaded via whichever protocol (http or https) the page itself is loaded. This removes the need to add any logic to support loading Tabzilla over both secure and insecure connections, thereby avoiding mixed-content warnings from the browser. 1.11.1 Requirements As the universal tab does inject HTML/CSS into the DOM, some there are some requirements that you must meet. • Background images must not be attached to the <body> element. • Absolutely positioned elements must not be positioned relative to the <body> element. • An element other than the <body> should add a 2 pixel white border to the top of the page (border-top: 2px solid #fff;) 32 Chapter 1. Contents playdoh Documentation, Release 1.0 Any background image or absolutely positioned element attached to the body element would not move with the rest of the contents when the tab slides open. Instead, any such background or element should be attached to anoter HTML element in the page (a wrapper div, for example). Note that this issue does not apply to solid background colors, or backgrounds that do not vary vertically (solid vertical stripes, for example). If jQuery is already included on the page, it will be used by Tabzilla. If jQuery is not already on the page, it will automatically be included after the page has loaded. 1.11.2 Translation Bar Tabzilla has an opt-in extension called Translation Bar that automatically offers a link to a localized page, if available, based on the user’s locale. It is intended to improve international user experience. Adding the Translation Bar extension to Tabzilla requires: 1. Include alternate URLs in the <head> element. For example: <link rel="alternate" hreflang="en-US" href="http://www.mozilla.org/en-US/firefox/new/" title="E <link rel="alternate" hreflang="fr" href="http://www.mozilla.org/fr/firefox/new/" title="Françai The Translation Bar alternatively detects available translations by looking for a language switcher like below, but implementation of alternate URLs is recommended also from the SEO perspective: <select id="language"><option value="en-US">English (US)</option></select> 2. Add the data-infobar attribute to the tab link, with the translation option: <a href="https://www.mozilla.org/" id="tabzilla" data-infobar="translation">mozilla</a> Note: Though the Translation Bar is currently implemented as an extension of Tabzilla, it might be moved to a standalone language utility in the future. 1.11.3 Update Bar This is another information bar intended to improve user security. It checks if the user is using the latest Firefox version, and if not, prompts the user to update the browser. Adding the Update Bar extension to Tabzilla requires: 1. Add the data-infobar attribute to the tab link, with the update option: <a href="https://www.mozilla.org/" id="tabzilla" data-infobar="update translation">mozilla</a> The value of the data-infobar attribute is the order of priority. You can enable both the Update Bar and Translation Bar as the example above, but one information bar will be shown at a time. 1.12 Mozilla.UITour 1.12.1 Introduction Mozilla.UITour is a JS library that exposes an event-based Web API for communicating with the Firefox browser chrome. It can be used for tasks such as opening menu panels and highlighting the position of buttons in the toolbar. It is supported in Firefox 29 onward. 1.12. Mozilla.UITour 33 playdoh Documentation, Release 1.0 For security reasons Mozilla.UITour will only work on white-listed domains and over a secure connection. The white-listed domains are https://www.mozilla.org and https://support.mozilla.org and the special about:home page. The Mozilla.UITour library is maintained on Mozilla Central. 1.12.2 Local development To develop or test using Mozilla.UITour locally you need to create some custom preferences in about:config. • browser.uitour.testingOrigins (string) (value: local address e.g. http://127.0.0.1:8000) • browser.uitour.requireSecure (boolean) (value: false) Note that browser.uitour.testingOrigins can be a comma separated list of domains, e.g. ‘http://127.0.0.1:8000, https://www-demo2.allizom.org‘ Important: Prior to Firefox 36, the testing preference was called browser.uitour.whitelist.add.testing (Bug 1081772). This old preference does not accept a comma separated list of domains, and you must also exclude the domain protocol e.g. https://. A browser restart is also required after adding a whitelisted domain. 1.12.3 JavaScript API registerPageID(pageId) Register an ID for use in Telemetry. pageId must be a string unique to the page: var pageId = ’firstrun-page-firefox-29’; Mozilla.UITour.registerPageID(pageId); showHighlight(target, effect) Highlight a button in the browser chrome. target is the string ID for the button and effect is the animation type: Mozilla.UITour.showHighlight(’appMenu’, ’wobble’); Target types: • ’accountStatus’ • ’addons’ • ’appMenu’ • ’backForward’ • ’bookmarks’ • ’customize’ • ’help’ • ’home’ • ’quit’ • ’search’ 34 Chapter 1. Contents playdoh Documentation, Release 1.0 • ’searchProvider’ (Firefox 33 and below) • ’searchIcon’ (Firefox 34 and above) • ’urlbar’ • ’loop’ • ’forget’ • ’privateWindow’ Effect types: • ’random’ • ’wobble’ • ’zoom’ • ’color’ • ’none’ (default) hideHighlight() Hides the currently visible highlight: Mozilla.UITour.hideHighlight(); showInfo(target, title, text, icon, buttons, options) Displays a customizable information panel pointing to a given target: var buttons = [ { label: ’Cancel’, style: ’link’, callback: cancelBtnCallback }, { label: ’Confirm’, style: ’primary’, callback: confirmBtnCallback } ]; var icon = ’//mozorg.cdn.mozilla.net/media/img/firefox/australis/logo.png’; var options = { closeButtonCallback: closeBtnCallback }; Mozilla.UITour.showInfo(’appMenu’, ’my title’, ’my text’, icon, buttons, options); Available targets: Any target that can be highlighted can have an information panel attached. Additional parameters: • title panel title (string). 1.12. Mozilla.UITour 35 playdoh Documentation, Release 1.0 • text panel description (string). • icon panel icon absolute url (string). Icon should be 48px x 48px. • buttons array of buttons (object) • options (object) buttons array items can have the following properties: • label button text (string) • icon button icon url (string) • style button style can be either primary or link (string) • callback to be excecuted when the button is clicked (function) • options (object) options can have the following properties: • closeButtonCallback to be excecuted when the (x) close button is clicked (function) hideInfo() Hides the currently visible info panel: Mozilla.UITour.hideInfo(); showMenu(target, callback) Opens a targeted menu in the browser chrome. Mozilla.UITour.showMenu(’appMenu’, function() { console.log(’menu was opened’); }); Available targets: • ’appMenu’ • ’bookmarks’ • ’searchEngines’ (only works for the old Search UI prior to Firefox 34) • ’loop’ (Firefox 35 and greater) Optional parameters: • callback function to be called when the menu was sucessfully opened. hideMenu(target) Mozilla.UITour.hideMenu(’appMenu’); Closes a menu panel. 36 Chapter 1. Contents playdoh Documentation, Release 1.0 previewTheme(theme) Previews a Firefox theme. theme should be a JSON literal: var theme = { "category": "iconURL": "headerURL": "name": "author": "footer": "previewURL": "updateURL": "accentcolor": "header": "version": "footerURL": "detailURL": "textcolor": "id": "description": }; "Firefox", "https://addons.mozilla.org/_files/18066/preview_small.jpg?1241572934", "https://addons.mozilla.org/_files/18066/1232849758499.jpg?1241572934", "Dark Fox", "randomaster", "https://addons.mozilla.org/_files/18066/1232849758500.jpg?1241572934", "https://addons.mozilla.org/_files/18066/preview.jpg?1241572934", "https://versioncheck.addons.mozilla.org/en-US/themes/update-check/18066", "#000000", "https://addons.mozilla.org/_files/18066/1232849758499.jpg?1241572934", "1.0", "https://addons.mozilla.org/_files/18066/1232849758500.jpg?1241572934", "https://addons.mozilla.org/en-US/firefox/addon/dark-fox-18066/", "#ffffff", "18066", "My dark version of the Firefox logo." Mozilla.UITour.previewTheme(theme); resetTheme() Removes the previewed theme and resets back to default: Mozilla.UITour.resetTheme(); cycleThemes(themes, delay, callback) Cycles through an array of themes at a set interval with a callback on each step: var themes = [ ... ]; var myCallback = function () { ... }; Mozilla.UITour.cycleThemes(themes, 5000, myCallback); • themes (array) • delay in milliseconds (number) • callback to excecute at each step (function) getConfiguration(type, callback) Queries the current browser configuration so the web page can make informed decisions on available highlight targets. Available type values: 1.12. Mozilla.UITour 37 playdoh Documentation, Release 1.0 • ’sync’ • ’availableTargets’ • ’appinfo’ • ’selectedSearchEngine’ • ’loop’ Other parameters: • callback function to excecute and return with the queried data Specific use cases: If ’sync’ is queried the object returned by the callback will contain an object called setup. This can be used to determine if the user is already using Firefox Sync: Mozilla.UITour.getConfiguration(’sync’, function (config) { if (config.setup === false) { // user is not using Firefox Sync } }); If ’availableTargets’ is queried the object returned by the callback contain array called targets. This can be used to determine what highlight targets are currently available in the browser chrome: Mozilla.UITour.getConfiguration(’availableTargets’, function (config) { console.dir(config.targets); }); If ’appinfo’ is queried the object returned gives information on the users current Firefox version. Mozilla.UITour.getConfiguration(’appinfo’, function (config) { console.dir(config); //{defaultUpdateChannel: "nightly", version: "36.0a1"} }); The defaultUpdateChannel key has many possible values, the most important being: • ’release’ • ’beta’ • ’aurora’ • ’nightly’ • ’default’ (self-build or automated testing builds) Important: appinfo is only available in Firefox 35 onward. If ’selectedSearchEngine’ is queried the object returned gives the currently selected default search provider. Mozilla.UITour.getConfiguration(’selectedSearchEngine’, function (data) { console.log(data.searchEngineIdentifier); // ’google’ }); Important: selectedSearchEngine is only available in Firefox 34 onward. If ’loop’ is queried the object returns the boolean value for the ’loop.gettingStarted.seen’ preference. 38 Chapter 1. Contents playdoh Documentation, Release 1.0 Mozilla.UITour.getConfiguration(’loop’, function (data) { console.log(data.gettingStartedSeen); // true }); Important: loop is only available in Firefox 36 onward. setConfiguration(name, value); Sets a specific browser preference using a given key value pair. Available key names: • ’Loop:ResumeTourOnFirstJoin’ Specific use cases: Setting the value for ’Loop:ResumeTourOnFirstJoin’ will enable Firefox to resume the FTE tour when the user joins their first conversation. Mozilla.UITour.setConfiguration(’Loop:ResumeTourOnFirstJoin’, true); Note: Don’t try setting this value to false. The current Hello code in Firefox handles when false should be set, and will actually set this value to true regardless whenever it is called. This will likely lead to unexpected results. Important: setConfiguration(’Loop:ResumeTourOnFirstJoin’, ...) is only available in Firefox 35 onward. showFirefoxAccounts(); Allows a web page to navigate directly to about:accounts?action=signup Mozilla.UITour.showFirefoxAccounts(); Important: showFirefoxAccounts() is only available in Firefox 31 onward. resetFirefox(); Opens the Firefox reset panel, allowing users to choose to reomve add-ons and customizations, as well as restore browser defaults. Mozilla.UITour.resetFirefox(); Important: showFirefoxAccounts() is only available in Firefox 35 onward. addNavBarWidget(target, callback); Adds an icon to the users toolbar • target can be an highlight target e.g. forget (string) • callback to excecute once icon added successfully (function) 1.12. Mozilla.UITour 39 playdoh Documentation, Release 1.0 Mozilla.UITour.addNavBarWidget(’forget’, function (config) { console.log(’forget button added to toolbar’); }); Important: Only available in Firefox 33.1 onward. setDefaultSearchEngine(id); Sets the browser default search engine provider. • id string identifier e.g. ‘yahoo’ or ‘google’. Mozilla.UITour.setDefaultSearchEngine(’yahoo’); • Identifiers for en-US US/searchplugins/list.txt builds: https://mxr.mozilla.org/mozilla-release/source/browser/locales/en- • Identifiers for other locales: https://mxr.mozilla.org/l10n-mozilla-release/find?string=browser%2Fsearchplugins%2Flist.txt Important: Only available in Firefox 34 onward. setSearchTerm(string); Populates the search UI with a given search term. • string search term e.g. ‘Firefox’ Mozilla.UITour.setSearchTerm(’Firefox’); Important: Only available in Firefox 34 onward. openSearchPanel(callback); Opens the search UI drop down panel. • callback function to excecute once the search panel has opened Mozilla.UITour.openSearchPanel(function() { console.log(’search panel opened’); }); Important: Only available in Firefox 34 onward. setTreatmentTag(name, value); Sets a key value pair as a treatment tag for recording in FHR. • name tag name for the treatment • value tag value for the treatment 40 Chapter 1. Contents playdoh Documentation, Release 1.0 Mozilla.UITour.setTreatmentTag(’srch-chg-action’, ’Switch’); Important: Only available in Firefox 34 onward. getTreatmentTag(name, callback); Retrieved the value for a set FHR. treatment tag. • name tag name to be retrieved • callback function to execute once the data has been retrieved Mozilla.UITour.getTreatmentTag(’srch-chg-action’, function(value) { console.log(value); }); Important: Only available in Firefox 34 onward. ping(callback); Pings Firefox to register that the page is using UiTour API. • callback function to execute when Firefox has acknowledged the ping. Mozilla.UITour.ping(function() { console.log(’UiTour is working!’); }); Important: Only available in Firefox 35 onward. observe(listener, callback); Register to listen for Firefox Hello events. • listener event handler for receiving Hello events • callback function to execute when event listener has been registered correctly Mozilla.UITour.observe(function(event, data) { console.log(event); console.log(data); }, function () { console.log(’event listener registered successfully’); }); Event types: • ’Loop:ChatWindowOpened’ - User opens the chat window. • ’Loop:ChatWindowClosed’ - User closes the chat window. • ’Loop:ChatWindowShown’ - User expands the chat window (also fires when chat window is opened). • ’Loop:ChatWindowHidden’ - User hides the chat window. • ’Loop:ChatWindowDetached’ - User detaches the chat window. 1.12. Mozilla.UITour 41 playdoh Documentation, Release 1.0 • ’Loop:IncomingConversation’ - User has an incoming conversation. Event will have data boolean value conversationOpen set to true or false depending on if the chat window is open or not. • ’Loop:RoomURLCopied’ - User clicks the copy button to share a chat URL. • ’Loop:RoomURLEmailed’ - User clicks the email button to share a chat URL. Note: UiTour can only create a single listener that is responsible for handling all event types. It is not currently possible to listen for only specific event types. To unbind listening for events, you can do: Mozilla.UITour.observe(null); Important: Only available in Firefox 35 onward. 42 Chapter 1. Contents
© Copyright 2024