JavaScript Binding

Getting Out of Binding Situations is an old article, but worth the read.

Posted in Uncategorized | Leave a comment

AngularJS ngLocale Error

AngularJS, like all frameworks has its warts. Its documentation is spartan but adequate for most purposed.

When AngularJS encounters an error, it tends to cough up a messy hairball of unhelpful information.

For example,

[$injector:nomod] Module 'ngLocale' is not available! You either misspelled the module name or forgot to load it. If registering a module ensure that you specify the dependencies as the second argument. http://errors.angularjs.org/1.2.7/$injector/nomod?p0=ngLocale

By all rights I read this as a failure to load ngLocale. It took me some backtracking to determine that the cause of the error has nothing to do with ngLocale.

The Cause

I simply attempted to use a service that exists in a module that hasn’t been loaded. Adding the module to the SAP dependency list resolved the error.

My druthers would be an error along the line of “Dependency ‘foo’ not found.”

 

Posted in Uncategorized | 4 Comments

Adding Sphinx Documentation to a Django Project

I hand not used Sphinx to generate documentation for a Django project before. Unfortunately I found the documentation web to be in the no man’s land between cryptic and sparse. After a number of mistakes, I was successful in producing auto-generated documentation from the Python code.

These are my notes for creating a basic set of documentation derived from the source code.

Caveat

I would prefer to keep all of Sphinx’s files in a subdirectory relative to the project root, but it appears to my untrained eye that Sphinx requires its configuration file and root document in the project root. Sphinx was happy to put its intermediaries and final product into a subdirectory, however.

Initial Installation

Running the Sphinx configuration script was straightforward. By default the autodoc (automatic documentation generation) is not enabled. Be sure the answer “y” to it.

Because I was trying (unsuccessfully) to get Sphinx to operate exclusively out of a subdirectory, I had to manually move the conf.py and index.rst files to the project root, then hack the config file so that templates_path, exclude_patterns, and html_static_path contained the subdirectory path.

Only at this point was I able to perform a build on this bare-bones set of documentation.

Overall Hierarchy

The structure that I got to work was a Sphinx index.rst file in the project root and every Python package directory. This forms a tree of index.rst files corresponding to packages.

Contents of the index.rst File

Header

Every index.rst file must contain a header. For example:

Foo
===

Sub-Packages

Each index.rst file contains the following if the package contains a sub-package:


.. toctree::
    :maxdepth: 2

    {sub-package-name-1}/index.rst
    {sub-package-name-2}/index.rst
    …
    {sub-package-name-n}/index.rst

Indentation is significant in a Python sense. Additionally, blank lines are significant.

Modules

Each index.rst file contains the following for each Python module in the same directory as the index.rst file:


.. automodule:: {full Python module name}
    :members:

Remember to ensure this is isolated from its neighbours by a blank line.

The autodoc process actually loads the Python module to perform introspection. From what I understand in the Sphinx documentation the full Python module name is required. For example, given the following file hierarchy:


package1/__init__.py
package1/foo.py
package1/package2/__init__.py
package1/package2/bar.py
package1/package2/index.rst
package1/index.rst

The Sphinx file package1/index.rst would contain:


.. automodule:: package1.foo
    :members:

The Sphinx file package1/package2/index.rst would contain:


.. automodule:: package1.package2.bar
    :members:

Non-Generated Documentation

Documentation that is not automatically generated can be kept in subdirectories and included in the ..toctree:: section in the same manner as sub-package .rst files are.

Update

I encountered a nice write-up here, which includes much more helpful information that I provide in these notes.

Posted in Programming | Tagged , , , , , | Leave a comment

Resolving a Django “Memory Leak” Problem on Long-Running Intensive Processes

Data conversions tend to be straightforward but detail-oriented processes that push systems in unexpected places. When importing several hundred thousand records into a Django development environment — entailing reshaping a number of important data sets — memory use continually escalated until the system slowed to a crawl.

The root cause is the environment is in debug mode, and Django connections log the queries that are executed. The import executes millions of database statements and runs for nearly a half hour. I prefer to not continually fiddle with the configuration file in my development environment, so  I did the following.

For any given data dump, every 60 seconds it prints a status of its progress: number of records completed, total number of records in the dump, and percent completed. The SQL statement log is not interesting, and is merely consuming memory, so it’s flushed thusly:

from django import db
...
db.reset_queries()

This has resolved the creeping memory problem. If there were any other unusual circumstances requiring releasing resources, this would be an opportune time.

 

 

Posted in Programming | Tagged , , , , , | Leave a comment

Going the Extra Mile on Data Coversions

Data conversions are not the most exciting types of projects, but I find them interesting because each is unique, and I enjoy the challenge of doing a superior job fitting old data into new systems. It’s normal to encounter un- or poorly-constrained text fields that have accumulated a great deal of cruft.

I dislike the attitude of merely dumping raw data into the new system and letting the user hold their noses at the digital sewage. My attitude is that the machine should conform to the needs of the user. Time spent viewing dirty string values and cleaning them up with some logic and a series of regular expressions turns digital sewage into useful relevant information, and much happier users.

 

Posted in Programming | Leave a comment

Zen of Python

Though it’s not the first language I reach for, Python is a language with many strengths, and I’m pleased it’s in my toolbox.

Using Python goes beyond its syntax, however. Besides the style guide in PEP 8, there is an accompanying guideline on how to use the language, æsthetically summarized in PEP 20. This is baked into the language as an easter egg — at the command line type:

import this

The Zen of Python

Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren’t special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one– and preferably only one –obvious way to do it.
Although that way may not be obvious at first unless you’re Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it’s a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea — let’s do more of those!

Posted in Programming | Tagged , , | Leave a comment

Understanding POST vs. PUT (in RESTful Web Sites)

A common point of focus between POST and PUT (which I find technically correct but unsatisfying) is that PUT is idempotent — repeating the PUT has the same effect every time (HTTP/1.1 §9.1.2).

The reason I find the answer unsatisfying is twofold. First, It’s a rare word even as jargon so it doesn’t facilitate communication. Second, it assumes a common knowledge of the design vision undergirding the web. In practical experience, I find a surprising number of professionals who have never given the subject much thought. This is not to denigrate anyone — as in most human affairs, practical daily tasks generally do not entail active consideration of relevant larger principles. We have work to do, after all.

Universal Resources

While quotidian use of the web revolves around browsing web sites and watching funny cat videos, there is a design framework underneath. RFC 1630 reminds us that it’s not just web pages with content to be consumed — there are a multitude of Internet-accessible resources. Consider the RFC’s description of a URL not as a web page address but as “an address which maps onto an access algorithm using network protocols”.

In other words, a URL is an address to access an algorithm. HTTP is a protocol — a formal method for accessing an algorithm.

The RESTful paradigm leverages this concept of accessing algorithms so that data can be created, retrieved, updated, and removed (the so-called CRUD paradigm). Retrieval is performed using GET, and removal is performed using DELETE.

PUT vs. POST

There is some debate about PUT vs. POST for creation, but I think that the W3C is fairly clear.

When a client uses PUT, it is stating “please store the following data under this URL”. Accessing the same URL should return the data. (The W3C steers clear of how this might affect the state of the server or the data). Conceptually, GET and PUT are analogous to read and write.

Of course, we’re not always writing to a known location. Appending a log, posting a new article to a site, sending mail, and recording data to a database with an autoincrementing primary key are common examples of data being handed over to the server without the assumption or knowability of the final URL.

POST simply requests “that the origin server accept the entity enclosed in the request”. If the POST results in data that can be accessed by a new URL, the server should return 201 (Created) and indicate the new location using, for example, a Location header.

Which to Use?

In my judgement, if the client know exactly where to record the data, PUT is the appropriate semantic, otherwise use POST.

For the work I do, CRUD — the create, read, update, delete cycle — most appropriately maps to POST, GET, PUT and DELETE because the client app does not know the final URL of the data it’s providing. It expects the server to handle the details of logging and database identifiers.

All that being said, if you only use POST requests, or only use PUT requests, there is no technical error which will cause the world to catch fire. Development frameworks universally support POST, but PUT is not always well-supported, requiring coding glue to be written. In this case I find using POST in a PUT semantic is entirely reasonable.

In the end I would say to understand the intent of the standards, and document what you’re doing.

Posted in Programming | Tagged , , , , | Leave a comment