Das Problem:

You have a field with 246 items to chose from … and you’re asking your users to scroll through them, hold control down, scroll to the next one, ups, forgot the control, start again … grrr.  Not pretty.

Do not fret.  crispy-forms and chosen.js have come to the rescue.  Here’s how it works and how simple it is:

1. Check out a chosen example to see how multi select works.

2. in your crispy form inside your form class add a line like:

Field('countries_of_operation', css_class='chosen', ),

This should render a form that looks something like:

<select multiple="multiple" class="selectmultiple chosen" 
id="id_countries_of_operation" name="countries_of_operation">

<option value="247">Antigua and Barbuda</option>

<option value="248">Algeria</option>

<option value="249">Azerbaijan</option>

etc ...

3. Add a little CSS and JS at the bottom of your page:

<script src="//ajax.googleapis.com/ajax/libs/jqueryui/1.10.2/jquery-ui.min.js">
</script>

<script type="text/javascript" 
src="{{ STATIC_URL }}js/chosen/chosen.jquery.min.js">
</script>

<link rel="stylesheet" href="{{ STATIC_URL }}js/chosen/chosen.css" 
type="text/css" media="screen" />

<script type="text/javascript">  $(".chosen").chosen();

</script>

4. Try it out!

In our search engine for renewables, we are running a bunch of PostGIS queries, e.g. to determine the wind speed at a user’s location. In our alpha search engine, we noticed that the PostGIS requests caused the longest query time of all 25 queries.

While experimenting with PostGIS, we discovered a nice time saver: In the alpha version, we translated the CSV data of measurement points (lat, lng, wind speed) into a shapefile with geometry points representing every measurement in the UK. The result looked like below (Southern England with a zoom on the Isle of Wight):

We extracted the data_point for a given location with the raw SQL statement in Django/Python:

We used the raw SQL statement in Django, since it provided the fastest query response. But it was still almost 300ms. Not enough!

So we looked for an alternative solution based on polygon intersect. Instead of looking for the closest data point to the geographic coordinate of the user’s location, we are now searching for the polygon which contains the user’s coordinate. Thanks to the WKT technology, intersect lookups run amazingly fast - in our case in 3ms.

If you have a shapefile with the point geometries (you can use QGIS to convert your CVS measurement data into shapefiles), you can use the QGIS geoprocessing function Voronoi Polygon (Vector | Geometry Tools | Voronoi Polygons) to convert the points into polygons for each measurement point/value. The result is a fancy British spider web:

PostGIS queries based on the polygon intersects can be done as follows:

data_point = WindData.objects.get(geom__intersects = location)

Since we do not have to sort the locations of the data points to find the closest data point anymore, the intersect query runs much faster. The WKT technology allows a hash comparison to determine if a point is part of a polygon or not. This amazing development provides the great speed increase.

The performance comparison in the Django shell says it all:

Nice reduction of almost 99% in the query time.

Say you want to send a friend a patch, or perhaps you often apply a change back and forth (say like preparing a file for production, and getting it back to testing). In our case base.html uses less.js to render the CSS during development, but in production it calls instead a minified CSS. So to convert base.html between the production and dev versions we created a patch. This is how you do it:

1. Modify the file

git diff dir/thefile.html --no-prefix > patch_thefile_from_dev2prod

The next time you need to change the file from dev to prod you simply apply the patch:

patch -p0 < patch_the_file_from_dev2prod

Ok, so here’s the third part of our series on our git-workflow (see part-1, part-2). We are 3 or 4 people working with git on a django project. So, what happens when one (and just one) of us needs to develope a small feature independently? This could be an extra form, an additional login option, a new field in the model.py or something that one person can tackle in 1 to 3 days.

Update the local database:

with a fab command that will copy the latest prod database to your local machine (see part-2 for details),

$ fab db_cp_prod2dev

then start a new branch for your feature

$ gpu

$ co -b dev_footer_menu_update

*remember I’m using the alliases introduced in the last post (gpu = git pull, co = git checkout, and co -b is like git branch branchname + git checkout branchname all in one). This takes me straight into the new branch as can be seen with:

$ git branch 

* dev_footer_menu_update

  dev_some_other_feature

  master

Great. so you create a wonderful new footer, update the stuff, stage your changes often (with git add -u) and commit at the end of the day.

NOTE:  If you had some files in master which you didn’t commit (some unfinished quick bugfix) you will need to go back to master and add/commit those before you continue on the branch.  Otherwise, doing rebase, merge, etc can give you problems.  So for example if you’ve worked on footer.html only in your branch, but your get:

$ git status 

# On branch dev_footer_menu_update

# Changes not staged for commit:

#   (use "git add ..." to update what will be committed)

#   (use "git checkout -- ..." to discard changes in working directory)

#
#       modified:   fabfile.py

#       modified:   mysite/assets/bootstrap/somestyle.less

#       modified:   mysite/templates/footer/footer.html

You should go back to master and finish the somestyle.less and fabfile.py first (obviously there are ways around this with git, but it will make it easier to follow these rules).  If it takes longer, it should have been in a branch to insure its independence.  Ok, so we finish the work on footer.html and:

$ git status 

# On branch dev_footer_menu_update

# Changes not staged for commit:

#   (use "git add ..." to update what will be committed)

#   (use "git checkout -- ..." to discard changes in working directory)

#       modified:   mysite/templates/footer/footer.html

$ ga -u # adds the files we modified, here footer.html

$ gcm "added the suppliers in the footer of the home template"

You go to bed, wake up in your polyphasic sleep, back to bed … and back to the computer. Oh! You forgot to style the footer.  Back to work.  Since we work accross 3 timezones, someone probably messed up your feature touching another file. To avert disaster you pull the changes that others have commited before  :

$ co master   # means: git checkout master

$ gpu   # means: git pull

$ co dev_footer_menu_update

$ git rebase master
First, rewinding head to replay your work on top of it...
Applying: added the suppliers in the footer of the home template

Rebase here will merge the stuff from master which other people have done into your branch.  So you can work on a branch that has the latest upstream files.  This may result in some conflicts, but it’s better to resolve those conflicts in your branch than later in master.  Once you’ve done the rebase in your branch, merging your branch into master will be easy and painless (because you already have the latest master in your branch except for your additions).  

So for example, after I commited a27aa52, I branched, worked and commited 18b831f. Once I rebase, I get the commits from my colleague Djangonaut Hannes into my branch: that is 6d412e8, 50c82fa, 43645e4.

by the way, that log is out “git log pretty”, 

alias glp='git log --pretty=format:"%Cred%h%Creset - %Cgreen%an%Creset, %ar : %s"'

Ok, so we rebase to get other people’s updates (or merge if there are too many conflicts). Now we work some more on the styles, check we are on the right branch:

$ gb
* dev_footer_menu_update

dev_some_other_feature

master 

TEST, add, commit, and merge:

$ ga -u  

$ gcm "updated footer, now with classy styles!"
$ co master
$ git merge dev_footer_menu_update

And finally delete the branch:

$ git branch -d dev_footer_menu_update

So to recap these are the steps:

• update the database
• pull, finish and commit any work on master
• create a branch, commit often
• rebase our branch in case master is evolving without us
• merge and delete the branch.

Now, that’s a lot of typing, so let’s review how we can do this with  aliases and fab functions:

$ gpu

$ fab db_cp_prod2dev

$ co -b dev_footer_menu_update

{ WORK GOES HERE}

$ ga -u

$ gcm "some interesting commit"

$ fab update_branch

{ MORE WORK GOES HERE}

$ ga -u

$ gcm "some interesting commit"

$ fab end_branch

Where update_branch and end_branch do the following:

When all this is done, we can of course

fab update_testing

send an email, let the others know, have a look, run tests, and:

fab update_dev

when ready. For a graphical representation of the workflow check out the next post.

Note that sending an email (or hipchat, or equivalent) after testing is very important. Otherwise, it could happen that another developer pushes to the repository, updates testing and finds an error. But without communication, if you are unaware of that error you might update master incorporating your last version (and his last version which has errors). It would be possible to build a more fool-proof workflow to take care of this, but it’s a complexity tradeoff, and communicating won’t hurt either!

Before creating a branch, doing a quick fix or messing with the models, we need to make sure our database is the same as the production one.

To do so we have a single bash (or fab) script which does:

This can all be summarized in a fab command which takes into account different OS (linux, mac), different absolute directories, and different naming of files (see fabfile.py in gist)

Ok, so we have one of those (Bfx), Urgent but simple bugfix. 

If our database is out of date we can,

$ fab db_cp_prod2dev

in other words “database copy from production to development” (see gist above).  And because it’s just a quick fix, we’ll stay on the master branch:

$ gb
  dev_some_feature
* master

Note that I will use a few git aliases to save my typing, in particular I have a ~/.bash_aliases with:

So, work, work, work, fix, fix, fix.

$ ga -u
$ gcm "quick fix, the margin in the CSS had gone cookoo, fixed .home-box class"
$ gp

Then send to testing

$ fab update_testing

and similarly send to production when ready

$ fab update_prod

Here’s a graphical overview of the whole thing:

And that’s it for bug-fixing alone.  Next part, developing a small feature in a branch.

This will be a series of exploratory articles on how to do django development in a team of 3 using git.  Suggestions from the readers are more than welcome, but the solutions have to be cheap and fast as we’re a bootstraped startup.

Setting the stage:

www.example.com (production)

• prod_db backed up automatically and regularly to dropbox and to a git repository with a cron job.
• media files, served by a static nginx app and lives in a sub directory /media/ of the above nginx.  Also compressed and backed up regularly to a dropbox account. 
• git repository of the django app, and dedicated virtual environment with requirements.txt tracked in the git repository.
• static files (fonts, img, css, js) are in the django project under mysite/assets/ and are served by an nginx static app which has symbolic links to mysite/assets/.

www.top-secret-testing-url.com (testing)

• testing_db  is regularly copied from prod_db, but sometimes diverges for testing. It is also added regularly to the db_backup git repository with a cron job.
• media files, served by a static nginx app and lives in a sub directory testing/media/ of the above nginx. This is so we don’t pollute the /media/ production directory with testing uploads.
• git repository of the django app, and dedicated virtual environment with requirements.txt tracked in the git repository.
• static files (fonts, img, css, js) are in the django project under my_testing_site/assets/ and are served by an nginx static app which has symbolic links from /testing/ to my_testing_site/assets/.

localhost (local)

• dev_db  is regularly wiped out and a new copy is loaded from prod_db
• media files, served by django locally, can be loaded using the gziped backup file.
• git repository of the django app, and dedicated virtual environment with requirements.txt tracked in the git repository.
• static files (fonts, img, css, js) are in the django project under assets/ and are served by django.

The user cases:

We will explore a few cases that arise often:

(Bfx) Urgent but simple bugfix.  

• font end: wrong color, typo, simple css fix
• minor typo leading to 404 or 500
• Obvious isolated things that can be fixed in 5mn - 2h

(ft) Small feature branch (single dev). 

• adding a small extra functionality: ex login with fb when twitter and gmail login already work, or one extra field in a model.py.
• can be done/tested by one person in 1-2 days
• is a tick in a trello list

(FT) Feature branch. 

• adding a full functionality: ex. new model with templates and views, integrating a new django-package, solving deep performance issues, etc …
• could require collaboration of 2-3 people in 1 or 2 branches.
• Is a full trello card.
• can take 2 - 10 days.

Along this exploration we need to remember how to keep our databases in sync with each other (including south migrations in testing and dev branches), how to sync our work when we work in different or common git branches at different speeds, how to backup everything, and how to create simple fabric scripts to simplify our life

Django’s debug error messages are usually helpful and provide a good level of description. However, sometimes we encounter error messages, which “amaze” us. We would like to take the chance to summarise the results of our solutions here on this blog.

Today, we worked on a new online feature for re.nooble’s customers. While writing some classes in the particular `views.py` (yes, we implement class-based responses), we received a strange error message:

Error: ‘function’ object has no attribute ‘as_view’

That error usually is expressed if you try to override a class. That sometimes happens if you copy&paste code and forget to change e.g. the class name. But that was not the case today.

After a while of online error digging and code testing, we discovered that the `art` of decorating classes/methods changed with introducing class-based views.

However, this way of decorating methods can not be applied to classes.
If you apply `@login_required`to a class, you will receive the error message: ‘function’ object has no attribute ‘as_view’

So, how should you decorate classes in Django now?
For class-based views, you have two options of decorating your classes.

1) Decorating the URLconf
2) Decorating the class

Both options leads to the same result - restricting the access to a class only for logged users. The difference between the options is how the decorator is applied to the class instance.

Regarding the detailed implementation and the detailed descriptions, please have a look at the very helpful Django documentation for decorating classes.

At the moment, we are creating a few flatpages for our re.nooble site. Obviously, they are created on a testing/development machine and need to be transferred to the production environment before the release of the latest code feature.

The few lines below will manage the transfer easily.

Ok, so I have some django projects using the now ubiquitous twitter bootstrap, together with the fantastish Font-Awesome. No problem. And I like my css LESS flavoured, because … well, because who likes to write css rules over and over? So this results in a folder structure for the assets which looks something like this:

├── assets
│   ├── bootstrap
│   │   ├── docs
│   │   ├── img
│   │   ├── js
│   │   └── less
│   ├── css
│   ├── font
│   ├── img
│   └── js

To enable font-awesome you’ll have inside the less directory a file called font-awesome.less indicating the relative path to the font folder:

@FontAwesomePath:   "../../font";

because the font file lives two levels out. So during development I would call the less files from base.html simply as:

But when I ran the Makefile

lessc --compress ${BOOTSTRAP_LESS} > ${BOOTSTRAP_MIN}                                                                                                                               

This would keep the relative path

@FontAwesomePath:   "../../font";

however, the minified css file now lived only one level away from the font directory (as you can see above). So what was the obvious solution? Well … here’s my dumb way out: modify the Makefile

So to solve this problem, I modify the Makefile, use sed, test some regex, test regex some more, break, crash burn … and finally make it work. Was this a good solution? Well, two weeks later I realized there’s a much, much simpler solution: make a copy of the font/ directory in the bootstrap/ directory … like so:

├── assets
│   ├── bootstrap
│   │   ├── docs
│   │   ├── img
│   │   ├── font
│   │   ├── js
│   │   └── less
│   ├── css
│   ├── font
│   ├── img
│   └── js

so that:

@FontAwesomePath:   "../../font";

@FontAwesomePath:   "../font";

DUUUHH!

Anyway. I just thought I would share this in case some unsuspecting n00b is looking for regex-es to solve this little issue

Here’s the setup from 0 to 127.0.0.1:8000 in 30 seconds:

I assume you already have python (below version 3.0) in your machine. You also have virtualenv (you can apt-get install python-virtualenv ) and pip. 

After that it’s the usual: create some models, runserver and syncdb