Tag Archives: python

Django 1.7, PyCharm, and Apache

I’ve had some ongoing problems with PyCharm support for Django since the 1.7 release. Here is a summary.

WSGI Broke

The entire app failed to run under Apache. This caused some moments of terror. The WSGI file needed to be edited.

This code started on version 1.3. The old WSGI configuration worked until 1.7. This discussion thread described the behaviour, which bites projects started pre-1.4. See also Issue #23437 describes the WSGI problem, and the official release notes under app-loading changes.

The correct call to initialize WSGI is:

[python]
from django.core.wsgi import get_wsgi_application
application = get_wsgi_application()
[/python]

 

PyCharm Testing Broke

Tests in PyCharm broke. This a case of being bitten by older code again. Django uses the standard Python unittest module now. I had to go through dozens of files and tweak the import statements.

Update: LH asked for more information.

In my unit testing, I used the old-style Django test classes and had import statements like:

[python]
from django.test import TestCase
[/python]

which must be changed to:

[python]
from unittest import TestCase
[/python]

in order to use the built-in Python unit testing core.

PyCharm Django Console Broke

This is kind of annoying. The problem isn’t so much PyCharm itself but Django “fixed” a long-standing “bug”. Now what used to work fine doesn’t. Per the app-loading changes in the 1.7 release notes, one must, every time they start the Django console from within PyCharm, execute the following commands:

[python]
import django
django.setup()
[/python]

This became annoying enough I hacked the Django helper

/Applications/PyCharm.app/helpers/pycharm/django_manage_shell.py

and added the two lines to that file. Problem resolved.

 

Python 2.7, Django, and MySQL on OS X

For some reason, getting Python and MySQL talking on OS X has been an annoyance. These are my notes for getting the two to talk to each other in a Python 2.7 virtual environment for a Django project.

The Django 1.6 docs recommend using the MySQLdb package. Its installation uses the mysql_config executable.

I have the following set up:

  1. MySQL 5.6.16
  2. Python 2.7.3
  3. PyCharm 3.1.1, which was used to create
  4. a Python virtual environment with pip located in $HOME/upharm27

[code]
$ locate mysql_config

/usr/local/mysql-5.6.16-osx10.7-x86_64/bin/mysql_config

$ export PATH=$PATH:/usr/local/mysql-5.6.16-osx10.7-x86_64/bin
$ ~/upharm27/bin/pip install mysql-python
[/code]

I have gcc 4.7.2 available, but curiously, the installer gave the following message:

[code]
Installing collected packages: mysql-python
Running setup.py install for mysql-python
gcc-4.2 not found, using clang instead
[/code]

The install succeeded using clang, so it’s nothing more than a curiosity at this point.

I was able to verify that the package was installed with:

[code]
$ ~/upharm27/bin/python
Python 2.7.3 (v2.7.3:70274d53c1dd, Apr  9 2012, 20:52:43)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import MySQLdb
>>>
[/code]

 

Update

I just followed these notes using the latest XCode 5.1 on OS X 10.9 and the mysql-python install failed. Apparently the clang Apple ships produces errors on unknown flags by default. I was able do get around this by:

[code]
$ export CFLAGS=-Qunused-arguments
$ export CPPFLAGS=-Qunused-arguments
$ ~/upharm27/bin/pip install mysql-python
[/code]

Thanks to the good folks at StackOverflow. More details may be found here.

Python 2 to 3 Conversion Notes

A project that I’m working on is being produced with the intention that it will be in production for a very long time — 10+ years. Thus converting from Python 2.7 to Python 3.4 seems to be a wise option to explore.

These are some personal notes regarding the differences between the two versions, supplementing the excellent articles out there. These are based on examining diffs after running 2to3 on the code base.

xrange and range

Contrary to what is implied in many articles, the separate functionality in xrange and range has not been mysteriously merged.The Python 2.7 xrange has been renamed to range. The Python 2.7 range is gone completely. To get the same effect, use

[python light=”true”]

foo = list(range(1, 10))

[/python]

 

The 2to3 code tool did this correctly; seeing its effect merely clarified how Python 3 works.

__unicode__

I had to manually go through my objects and straighten out __str__ and __unicode__. This involved:

  1. Deleting __str__ methods which essentially duplicated __unicode__.
  2. Renaming __unicode__ to __str__.

__long__

I used __long__ in one object, and simply deleted it since under Python 3 it is a duplicate of __int__.

long() and int()

Similar to __long__, there was one instance where I simply deleted the code associated with long().

super()

I had to manually convert the Python 2.7

[python light=”true”]

super(ClassName, self)

[/python]

to

[python light=”true”]

super()

[/python]

 map()

2to3 attempts to convert map() statements to list comprehensions. It did well on most, but screwed up in two places. It might be wise to search for leftover map() statements.

Updates

14 May 2014
Added notes for map().

 

 

Django and AngularJS POST

I’m converting some code that uses jQuery to post data to Django over to AngularJS. The POSTed data wasn’t coming across as expected.

By default AngularJS and jQuery use two different methods of packaging data. According what I’ve gleaned from googling:

  • jQuery posts using a content type of application/x-www-form-urlencoded and serializes the data using the foo=bar&fuz=baz method.
  • AngularJS posts using a content type of application/json and serializes the data in the body.

Because this is an Angular single-page application, rather than using Django’s templating, I’m making calls to a web-based API (implemented in Django). I use a simple helper class for the repetitive task of extracting POSTed data, converting it to Python objects, and validating it.

I made a simple change to this object, and voilà — Django started receiving AngularJS data.

[python]
# CONTEXT:
# This function extracts the received data and
# returns it as a Python dict.
if ‘CONTENT_TYPE’ in request.META and request.META[‘CONTENT_TYPE’].startswith(‘application/json’):
return json.loads(request.body)
elif request.method == ‘POST’:
return request.POST.dict()
elif …
[/python]

If you wish to use Django’s unit testing to exercise this, use something like the following. Remember to send a JSON serialized string into post() as it will not do this for you.

[python]
from json import dumps
from django.test.client import Client


client = Client()
self.assertTrue(client.login(username=user, password=password))
response = client.post(url, dumps(data), content_type=’application/json’)
[/python]

Updates

2014-02-21
Fixed bug in sample code.
2014-06-05
Added unit testing example.

Adding Sphinx Documentation to a Django Project

I hand not used Sphinx to generate documentation for a Django project before. Unfortunately I found the documentation web to be in the no man’s land between cryptic and sparse. After a number of mistakes, I was successful in producing auto-generated documentation from the Python code.

These are my notes for creating a basic set of documentation derived from the source code.

Caveat

I would prefer to keep all of Sphinx’s files in a subdirectory relative to the project root, but it appears to my untrained eye that Sphinx requires its configuration file and root document in the project root. Sphinx was happy to put its intermediaries and final product into a subdirectory, however.

Initial Installation

Running the Sphinx configuration script was straightforward. By default the autodoc (automatic documentation generation) is not enabled. Be sure the answer “y” to it.

Because I was trying (unsuccessfully) to get Sphinx to operate exclusively out of a subdirectory, I had to manually move the conf.py and index.rst files to the project root, then hack the config file so that templates_path, exclude_patterns, and html_static_path contained the subdirectory path.

Only at this point was I able to perform a build on this bare-bones set of documentation.

Overall Hierarchy

The structure that I got to work was a Sphinx index.rst file in the project root and every Python package directory. This forms a tree of index.rst files corresponding to packages.

Contents of the index.rst File

Header

Every index.rst file must contain a header. For example:

[code]
Foo
===

[/code]

Sub-Packages

Each index.rst file contains the following if the package contains a sub-package:

[code]

.. toctree::
:maxdepth: 2

{sub-package-name-1}/index.rst
{sub-package-name-2}/index.rst

{sub-package-name-n}/index.rst

[/code]

Indentation is significant in a Python sense. Additionally, blank lines are significant.

Modules

Each index.rst file contains the following for each Python module in the same directory as the index.rst file:

[code]

.. automodule:: {full Python module name}
:members:

[/code]

Remember to ensure this is isolated from its neighbours by a blank line.

The autodoc process actually loads the Python module to perform introspection. From what I understand in the Sphinx documentation the full Python module name is required. For example, given the following file hierarchy:

[code]

package1/__init__.py
package1/foo.py
package1/package2/__init__.py
package1/package2/bar.py
package1/package2/index.rst
package1/index.rst

[/code]

The Sphinx file package1/index.rst would contain:

[code]

.. automodule:: package1.foo
:members:

[/code]

The Sphinx file package1/package2/index.rst would contain:

[code]

.. automodule:: package1.package2.bar
:members:

[/code]

Non-Generated Documentation

Documentation that is not automatically generated can be kept in subdirectories and included in the ..toctree:: section in the same manner as sub-package .rst files are.

Update

I encountered a nice write-up here, which includes much more helpful information that I provide in these notes.

Zen of Python

Though it’s not the first language I reach for, Python is a language with many strengths, and I’m pleased it’s in my toolbox.

Using Python goes beyond its syntax, however. Besides the style guide in PEP 8, there is an accompanying guideline on how to use the language, æsthetically summarized in PEP 20. This is baked into the language as an easter egg — at the command line type:

[python]
import this
[/python]

The Zen of Python

Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren’t special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one– and preferably only one –obvious way to do it.
Although that way may not be obvious at first unless you’re Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it’s a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea — let’s do more of those!

Installing Django on CentOS 6

Django is available as an EPEL package.

First, activate EPEL via

[code lang=”bash”]
$ sudo rpm -Uvh http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm
[/code]

or for 32-bin CentOS,

[code lang=”bash”]
$ sudo rpm -Uvh http://download.fedoraproject.org/pub/epel/6/i386/epel-release-6-8.noarch.rpm
[/code]

NOTE: That exact URL will change with time. I took it from the EPEL FAQ.

Next, install the Django package.

[code lang=”bash”]
$ sudo rpm install Django
[/code]

Ensure that Django is working.

[code lang=”python”]
$ python
Python 2.6.6. (r266:84292, Feb 22 2013, 00:00:18)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-3)] on linux2
Type "help", "copyright", "credits" or license" for more informaiton.
>>> import django
>>> print django.get_version()
1.4.5
>>> _
[/code]