Wednesday, September 23, 2009

Django|Custom Search 302

Custom Search returns homepage - Custom Search Help

Custom Search returns homepage Report abuse

nrhrehab
Level 1
3/18/09
I have 2 sites using the custom search -- one is working fine (nrhrehab.org); the other (nrhhealthtown.com) had been working, but failed at some point. The Search that is not working (on nrhhealthtown.com) redirects to the site's homepage when the submit button is hit. I did add Google Ananlytics to this site (but not the other) a few weeks ago. Both sites are registered under one account (nrhrehab)

All replies

omr
Level 3
3/18/09
When the browser attempts to open nrhhealthtown.com/search.htm with parameters specified in the URL, the web server issues a (type 302) redirect.  I think you may need to ask your web server administrator or web host tech support for help to correct the server configuration to prevent the redirect.

Saturday, September 12, 2009

科学: 经济物理学家解释沪市泡沫破裂预言

Solidot: 奇客的资讯,重要的东西

科学: 经济物理学家解释沪市泡沫破裂预言

matrix 发表于 2009年9月11日 16时12分 星期五   Printer-friendly   Email story
来自follow-the-party部门
7月中旬,瑞士苏黎世联邦理工学院经济物理学家Didier Sornette宣称,上海综合股票市场存在巨大泡沫,他声称泡沫的破灭日期发生在7月17日到27日之间。虽然沪市指数的暴跌并没有发生在27日之前,但整个8月份它确实出现了激烈的震荡,从8月初开始,股票指数下降了20%。 现在Sornette和同事发表了一篇预印本公开了他们预测市场泡沫的方法。他们的方法包含了泡沫理性预期的经济学理论,投资者和交易员的模仿和羊群行为(当个体无法理解经济的运行状况时,他们往往会追随他人,形成羊群),分歧现象和相转移的数学和统计物理学。研究人员表示利用这些方法,他们看到了股市明确的泡沫信号,因此预测沪市的超指数增长将会发生震荡。Sornette声称他们成功预测了去年石油泡沫的崩溃,以及2006年美国房地产泡沫的破灭。当然,泡沫的结束并不意味着会发生一场崩溃,它可能是从超指数增长变成一种市场动力学。

科学: 经济物理学家解释沪市泡沫破裂预言

Solidot: 奇客的资讯,重要的东西

科学: 经济物理学家解释沪市泡沫破裂预言

matrix 发表于 2009年9月11日 16时12分 星期五   Printer-friendly   Email story
来自follow-the-party部门
7月中旬,瑞士苏黎世联邦理工学院经济物理学家Didier Sornette宣称,上海综合股票市场存在巨大泡沫,他声称泡沫的破灭日期发生在7月17日到27日之间。虽然沪市指数的暴跌并没有发生在27日之前,但整个8月份它确实出现了激烈的震荡,从8月初开始,股票指数下降了20%。 现在Sornette和同事发表了一篇预印本公开了他们预测市场泡沫的方法。他们的方法包含了泡沫理性预期的经济学理论,投资者和交易员的模仿和羊群行为(当个体无法理解经济的运行状况时,他们往往会追随他人,形成羊群),分歧现象和相转移的数学和统计物理学。研究人员表示利用这些方法,他们看到了股市明确的泡沫信号,因此预测沪市的超指数增长将会发生震荡。Sornette声称他们成功预测了去年石油泡沫的崩溃,以及2006年美国房地产泡沫的破灭。当然,泡沫的结束并不意味着会发生一场崩溃,它可能是从超指数增长变成一种市场动力学。

Friday, September 11, 2009

杂志评世界最伟大城市:欧美包揽前五 悉尼第九(图)

杂志评世界最伟大城市:欧美包揽前五 悉尼第九(图) -6park.com

杂志评世界最伟大城市:欧美包揽前五 悉尼第九(图)

新闻来源: 国际在线 于September 10, 2009 22:44:51 敬请注意:新闻取自各大新闻媒体,观点内容并不代表本网立场!

线 www.6park.com

  日前,英国老牌生活杂志《消费导刊》组织专家评选出"世界最伟大的城市"榜单,并将其编纂成书,位列该榜单的共有75个城市。 www.6park.com

  据《泰晤士报》报道,这份榜单中的城市并非那些旅游胜地,而是被认为适宜人居又适宜工作的都市,也就是说,这些城市并不是那些只适合度假的地方,该项评估包括了城市里居民日常生活的方方面面。参与调查的城市居民被要求对自己每天的生活标准进行评估,如城市的建筑(都市风景)、艺术、文化、商业、餐饮、生活质量、国际形象等等。 www.6park.com

  这些评估数据随后被送到《消费导刊》的评委手中,经过他们的讨论和评估得出最终结果,下面是排在榜单前十位的城市。 www.6park.com

www.6park.com

1. 纽约,美国
www.6park.com

  这似乎并不出乎人们的意料。纽约总是排在各种城市榜单的首位,并几近成为国际化都市的典范。该评选最初开始于 2009年4月,当时这座老牌国际化都市正因金融危机的侵袭显得步履蹒跚,一些亚洲城市几乎要偷走纽约的皇冠。然而,经过各项指标的评估――建筑、艺术、文化、商业、餐饮、生活质量和国际形象,纽约还是让人很难找出缺点。 www.6park.com


www.6park.com

2. 伦敦,英国
www.6park.com

  虽然伦敦的市民已经逐渐丢掉了国家鼎盛时那种昂首阔步的气派,但就文化而言,伦敦仍旧吸引着世界的目光――它的剧院、歌舞、艺术、传统、文学、音乐、时尚,甚至还有电影产业。 www.6park.com



www.6park.com


3. 巴黎,法国
www.6park.com

  巴黎似乎还在仰仗曾经那个完美城市的余泽,但评估中我们还是能看到这座城市的浪漫与狂热。 www.6park.com

www.6park.com


www.6park.com


4. 柏林,德国
www.6park.com

  虽然柏林已经走入了新的千年,但仍带着上个世纪的汹涌澎湃。这座经历过毁灭与重生的城市依然自信,这里融汇了大胆的现代艺术和对艺术的尖刻批评。这里每年都举办令人炫目的电影节,同时也是世界顶级交响乐团的家园。 www.6park.com


www.6park.com


5. 巴塞罗那,西班牙

www.6park.com

  单从建筑上看,巴塞罗那就足以让人震撼。这里有加泰罗尼亚风情的华丽现代建筑,又有安东尼・高迪那历经百年仍未完成的宏伟建筑蓝图。 www.6park.com


www.6park.com


5.芝加哥,美国

www.6park.com

  也许这是最让人惊讶的结果之一,也会有人说居住在这里并不轻松。这里的冬天非常寒冷,有些地区甚至还存在着种族隔离的色彩。虽然芝加哥的名声并不好,但曾经居住在那里或经常造访这里的《消费导刊》的编辑们认为,这里其实具有超凡的文化魅力,尤其是它的建筑――出自这里的摩天大楼甚至成为20世纪建筑潮流的典范。而作为奥巴马大选时的竞选总部,这里更为美国送出第一位非洲裔总统。芝加哥还有可能成为奥运会的举办地,这都让这座城市变得可爱。 www.6park.com

www.6park.com


5. 东京,日本

www.6park.com

  一个城市所应该具备的基本要素:明亮的灯光、节奏紧张的步伐、惊人的消费量、现代化的科技、强劲的股市、街头文化、高楼大厦、上百万的人口……一切的一切,让东京在评选中获得很高的分数。 www.6park.com

www.6park.com


8. 伊斯坦布尔,土耳其

www.6park.com

  一座大而用心经营的城市,从城市的市场、酒吧、上下班的人群中就能感受到无穷的活力。伊斯坦布尔著名的美景在于跨越地理空间的博斯普鲁斯海峡和遗迹,尤其是那些清真寺的唤礼塔。这里最著名的是蓝色清真寺和拜占庭时期的壁画。当地的居民还会告诉你,这里的落日非常壮观而庄严。 www.6park.com

www.6park.com

9. 罗马,意大利

www.6park.com

  从斗兽场到国立当代艺术博物馆,罗马从不惧怕陈述和炫耀自己的历史文化。虽然城市的大多能量来自两千多年前,但历史和现代气息在这里交融,让城市继续迸发着力量。在匹萨饼的香气中,在科林斯风格的石柱、花园和文艺复兴风格的豪华宫殿所组成的背景下,高雅的剧院里仍在认真地排练着。这座七丘山上的永恒之城唯一的美中不足在于它恼人而效率低下的交通。 www.6park.com


www.6park.com

9. 悉尼,澳大利亚

www.6park.com

  让悉尼大赚分数的是它多元的文化和健康的户外生活方式,这里的居民陶醉于清晨冲浪,这里有世界著名的海滩,优质的海边建筑,优秀的美食和精彩的咖啡馆文化。这里的夜生活五彩斑斓,新年的焰火表演让人艳羡。

Wednesday, September 9, 2009

Linux|"Argument list too long"

"Argument list too long": Beyond Arguments and Limitations

"Argument list too long": Beyond Arguments and Limitations

May 9th, 2002 by Alessandre S. Naro in

Four approaches to getting around argument length limitations on the command line.

At some point during your career as a Linux user, you may have come across the following error:

[user@localhost directory]$ mv * ../directory2 bash: /bin/mv: Argument list too long 

The "Argument list too long" error, which occurs anytime a user feeds too many arguments to a single command, leaves the user to fend for oneself, since all regular system commands (ls *, cp *, rm *, etc...) are subject to the same limitation. This article will focus on identifying four different workaround solutions to this problem, each method using varying degrees of complexity to solve different potential problems. The solutions are presented below in order of simplicity, following the logical principle of Occam's Razor: If you have two equally likely solutions to a problem, pick the simplest.

Method #1: Manually split the command line arguments into smaller bunches.

Example 1

[user@localhost directory]$ mv [a-l]* ../directory2 [user@localhost directory]$ mv [m-z]* ../directory2 

This method is the most basic of the four: it simply involves resubmitting the original command with fewer arguments, in the hope that this will solve the problem. Although this method may work as a quick fix, it is far from being the ideal solution. It works best if you have a list of files whose names are evenly distributed across the alphabet. This allows you to establish consistent divisions, making the chore slightly easier to complete. However, this method is a poor choice for handling very large quantities of files, since it involves resubmitting many commands and a good deal of guesswork.

Method #2: Use the find command.

Example 2

[user@localhost directory]$ find $directory -type f -name '*' -exec mv {} $directory2/. \; 

Method #2 involves filtering the list of files through the find command, instructing it to properly handle each file based on a specified set of command-line parameters. Due to the built-in flexibility of the find command, this workaround is easy to use, successful and quite popular. It allows you to selectively work with subsets of files based on their name patterns, date stamps, permissions and even inode numbers. In addition, and perhaps most importantly, you can complete the entire task with a single command.

The main drawback to this method is the length of time required to complete the process. Unlike Method #1, where groups of files get processed as a unit, this procedure actually inspects the individual properties of each file before performing the designated operation. The overhead involved can be quite significant, and moving lots of files individually may take a long time.

Method #3: Create a function. *

Example 3a

function large_mv () {       while read line1; do                 mv directory/$line1 ../directory2         done } ls -1 directory/ | large_mv 

Although writing a shell function does involve a certain level of complexity, I find that this method allows for a greater degree of flexibility and control than either Method #1 or #2. The short function given in Example 3a simply mimics the functionality of the find command given in Example 2: it deals with each file individually, processing them one by one. However, by writing a function you also gain the ability to perform an unlimited number of actions per file still using a single command:

Example 3b

function larger_mv () {       while read line1; do                 md5sum directory/$line1 >>  ~/md5sums                 ls -l directory/$line1 >> ~/backup_list                 mv directory/$line1 ../directory2         done } ls -1 directory/ | larger_mv 

Example 3b demonstrates how you easily can get an md5sum and a backup listing of each file before moving it.

Unfortunately, since this method also requires that each file be dealt with individually, it will involve a delay similar to that of Method #2. From experience I have found that Method #2 is a little faster than the function given in Example 3a, so Method #3 should be used only in cases where the extra functionality is required.

Method #4: Recompile the Linux kernel. **

This last method requires a word of caution, as it is by far the most aggressive solution to the problem. It is presented here for the sake of thoroughness, since it is a valid method of getting around the problem. However, please be advised that due to the advanced nature of the solution, only experienced Linux users should attempt this hack. In addition, make sure to thoroughly test the final result in your environment before implementing it permanently.

One of the advantages of using an open-source kernel is that you are able to examine exactly what it is configured to do and modify its parameters to suit the individual needs of your system. Method #4 involves manually increasing the number of pages that are allocated within the kernel for command-line arguments. If you look at the include/linux/binfmts.h file, you will find the following near the top:

/*  * MAX_ARG_PAGES defines the number of pages allocated for   arguments  * and envelope for the new program. 32 should suffice, this gives  * a maximum env+arg of 128kB w/4KB pages!  */ #define MAX_ARG_PAGES 32 

In order to increase the amount of memory dedicated to the command-line arguments, you simply need to provide the MAX_ARG_PAGES value with a higher number. Once this edit is saved, simply recompile, install and reboot into the new kernel as you would do normally.

On my own test system I managed to solve all my problems by raising this value to 64. After extensive testing, I have not experienced a single problem since the switch. This is entirely expected since even with MAX_ARG_PAGES set to 64, the longest possible command line I could produce would only occupy 256KB of system memory--not very much by today's system hardware standards.

The advantages of Method #4 are clear. You are now able to simply run the command as you would normally, and it completes successfully. The disadvantages are equally clear. If you raise the amount of memory available to the command line beyond the amount of available system memory, you can create a D.O.S. attack on your own system and cause it to crash. On multiuser systems in particular, even a small increase can have a significant impact because every user is then allocated the additional memory. Therefore always test extensively in your own environment, as this is the safest way to determine if Method #4 is a viable option for you.

Conclusion

While writing this article, I came across many explanations for the "Argument list too long" error. Since the error message starts with "bash:", many people placed the blame on the bash shell. Similarly, seeing the application name included in the error caused a few people to blame the application itself. Instead, as I hope to have conclusively demonstrated in Method #4, the kernel itself is to "blame" for the limitation. In spite of the enthusiastic endorsement given by the original binfmts.h author, many of us have since found that 128KB of dedicated memory for the command line is simply not enough. Hopefully, by using one of the methods above, we can all forget about this one and get back to work.

Notes:

* All functions were written using the bash shell.

** The material presented in Method #4 was gathered from a discussion on the linux-kernel mailing list in March 2000. See the "Argument List too Long" thread in the linux-kernel archives for the full discussion.

Tuesday, August 25, 2009

Java| Thread Dump

My Load Test » Java Thread Dump

Java Thread Dump

A Java thread dump is a way of finding out what every thread in the JVM is doing at a particular point in time. This is especially useful if your Java application sometimes seems to hang when running under load, as an analysis of the dump will show where the threads are stuck.

You can generate a thread dump under Unix/Linux by running kill -QUIT <pid>, and under Windows by hitting Ctl + Break.

A great example of where this would be useful is the well-known Dining Philosophers deadlocking problem. Taking example code from Concurrency: State Models & Java Programs, we can cause a deadlock situation and then create a thread dump.

Thursday, August 13, 2009

Django|SerializedDataField

Custom Fields in Django | David Cramer's Blog

I was helping someone today in the Django IRC channel and the question came across about storing a denormalized data set in a single field. Typically I do such things by either serializing the data, or by separating the values with a token (comma for example).

Django has a built-in field type for CommaSeparatedIntegerField, but most of the time I'm storing strings, as I already have the integers available elsewhere. As I began to answer the person's question by giving him an example of usage of serialization + custom properties, until I realized that it would be much easier to just write this as a Field subclass.

So I quickly did, and replaced a few lines of repetitive code with two new field classes in our source:

Update: There were some issues with my understanding of how the metaclass was working. I've corrected the code and it should function properly now.

SerializedDataField

This field is typically used to store raw data, such as a dictionary, or a list of items, or could even be used for more complex objects.

 from django.db import models   try:     import cPickle as pickle except:     import pickle   import base64   class SerializedDataField(models.TextField):     """Because Django for some reason feels its needed to repeatedly call     to_python even after it's been converted this does not support strings."""     __metaclass__ = models.SubfieldBase       def to_python(self, value):         if value is None: return         if not isinstance(value, basestring): return value         value = pickle.loads(base64.b64decode(value))         return value       def get_db_prep_save(self, value):         if value is None: return         return base64.b64encode(pickle.dumps(value))

SeparatedValuesField

An alternative to the CommaSeparatedIntegerField, it allows you to store any separated values. You can also optionally specify a token parameter.

 from django.db import models   class SeparatedValuesField(models.TextField):     __metaclass__ = models.SubfieldBase       def __init__(self, *args, **kwargs):         self.token = kwargs.pop('token', ',')         super(SeparatedValuesField, self).__init__(*args, **kwargs)       def to_python(self, value):         if not value: return         if isinstance(value, list):             return value         return value.split(self.token)       def get_db_prep_value(self, value):         if not value: return         assert(isinstance(value, list) or isinstance(value, tuple))         return self.token.join([unicode(s) for s in value])       def value_to_string(self, obj):         value = self._get_val_from_obj(obj)         return self.get_db_prep_value(value)

Friday, August 7, 2009

Django|Tips to keep your Django/mod_python memory usage down

Tips to keep your Django/mod_python memory usage down - WebFaction

Tips to keep your Django/mod_python memory usage down

Updated Jan 28 at 04:44 CDT (first posted May 30 at 09:57 CDT) by Remi in Django, Memory, Tips  - 16 comment(s)

Most people manage to run their Django site on mod_python within the memory limits of their "Shared 1" or "Shared 2" plans but a few people are struggling to stay within the limits.

So here are a few tips that you can use to try and keep your memory usage down when using Django on mod_python:

  • Make sure that you set DEBUG to False in settings.py: if it isn't, set it to False and restart apache. Amongst other things, DEBUG mode stores all SQL queries in memory so your memory usage will quickly increase if you don't turn it off.
  • Use "ServerLimit" in your apache config: by default apache will spawn lots of processes, which will use lots of memory. You can use the "ServerLimit" directive to limit these processes. A value of 3 or 4 is usually enough for most sites if your static data is not served by your Django instance (see below).
  • Check that no big objects are being loaded in memory: for instance, check that your code isn't loading hundreds or thousands of database records in memory all at once. Also, if your application lets people download or upload big files, check that these big files are not being loaded in memory all at once.
  • Serve your static data from our main server: this is a general advice for all django sites: make sure that your static data (images, stylesheets, ...) is served directly by our main apache server. This will save your Django app from having to serve all these little extra requests. Details on how to do that can be found here and here.
  • Use "MaxRequestsPerChild" in your apache config: sometimes there are some slow memory leaks that you can't do anything about (they can be in the tools that you use themselves for instance). If this is the case then you can use the "MaxRequestsPerChild" to tell apache to only serve a certain number of requests before killing the process and starting a fresh one. Reasonable values are usually between 100 and 1000. Another more extreme/uglier version of this technique is to setup a cronjob to run "stop/start" once in a while.
  • Find out and understand how much memory you're using: to find out what your processes are and how much memory they're using, you can run the "ps -u <username> -o pid,rss,command" command, like this:
    1 [testweb14@web14 bin]$ ps -u testweb14 -o pid,rss,command 
    2PID RSS COMMAND 
    323111 1404 -bash 
    427988 3848 /home/testweb14/webapps/django/apache2/bin/httpd -f /home/testweb14/webapps/django/apache2/conf/httpd.c 
    5 27989 10312 /home/testweb14/webapps/django/apache2/bin/httpd -f /home/testweb14/webapps/django/apache2/conf/httpd. 
    627990 9804 /home/testweb14/webapps/django/apache2/bin/httpd -f /home/testweb14/webapps/django/apache2/conf/httpd.c 
    7 28078 760 ps -u testweb14 -o pid,rss,command 
    8[testweb14@web14 bin]$ 
    view plain | print | ?
    As you can see we have three "httpd" processes running that use respectively 3848KB, 10312KB and 9804KB of memory (there are various ways to interpret the memory used by a process on Linux and we have chosen to use the "Resident Set Size" (RSS) or your processes).

    The first one is the apache "supervisor" and the other two are the "workers" (in this example, "ServerLimit" is set to 2). The memory used by the supervisor usually doesn't change too much, but the memory used by the workers can increase greatly if you have bad memory leaks in your application.

    So the total memory used by our Apache/django instance in this example is 3848KB + 10312KB + 9804KB = 23MB.

Wednesday, August 5, 2009

Django|Speed up with NginX, Memcached, and django-compress

How to Speed up Your Django Sites with NginX, Memcached, and django-compress | Code Spatter

How to Speed up Your Django Sites with NginX, Memcached, and django-compress

Posted on April 23rd, 2009 by Greg Allard in Django, Programming, Server Administration | View commentsComments

A lot of these steps will speed up any kind of application, not just django projects, but there are a few django specific things. Everything has been tested on IvyLees which is running in a Debian/Ubuntu environment.

These three simple steps will speed up your server and allow it to handle more traffic.

Reducing the Number of HTTP Requests

Yahoo has developed a firefox extension called YSlow. It analyzes all of the traffic from a website and gives a score on a few categories where improvements can be made.

It recommends reducing all of your css files into one file and all of your js files into one file or as few as possible. There is a pluggable, open source django application available to help with that task. After setting up django-compress, a website will have css and js files that are minified (excess white space and characters are removed to reduce file size). The application will also give the files version numbers so that they can be cached by the web browser and won't need to be downloaded again until a change is made and a new version of the file is created. How to setup the server to set a far future expiration is shown below in the lightweight server section.

Setting up Memcached

Django makes it really simple to set up caching backends and memcached is easy to install.

sudo aptitude install memcached, python-setuptools

We will need setuptools so that we can do the following command.

sudo easy_install python-memcached

Once that is done you can start the memcached server by doing the following:

sudo memcached -d -u www-data -p 11211 -m 64

-d will start it in daemon mode, -u is the user for it to run as, -p is the port, and -m is the maximum number of megabytes of memory to use.

Now open up the settings.py file for your project and add the following line:

CACHE_BACKEND = 'memcached://127.0.0.1:11211/'

Find the MIDDLEWARE_CLASSES section and add this to the beginning of the list:

    'django.middleware.cache.UpdateCacheMiddleware',

and this to the end of the list:

    'django.middleware.cache.FetchFromCacheMiddleware',

For more about caching with django see the django docs on caching. You can reload the server now to try it out.

sudo /etc/init.d/apache2 reload

To make sure that memcached is set up correctly you can telnet into it and get some statistics.

telnet localhost 11211

Once you are in type stats and it will show some information (press ctrl ] and then ctrl d to exit). If there are too many zeroes, it either isn't working or you haven't visited your site since the caching was set up. See the memcached site for more information.

Don't Use Apache for Static Files

Apache has some overhead involved that makes it good for serving php, python, or ruby applications, but you do not need that for static files like your images, style sheets, and javascript. There are a few options for lightweight servers that you can put in front of apache to handle the static files. Lighttpd (lighty) and nginx (engine x) are two good options. Adding this layer in front of your application will act as an application firewall so there is a security bonus to the speed bonus.

There is this guide to install a django setup with nginx and apache from scratch. If you followed my guide to set up your server or already have apache set up for your application, then there are a few steps to get nginx handling your static files.

sudo aptitude install nginx

Edit the config file for your site (sudo nano /etc/apache2/sites-available/default) and change the port from 80 to 8080 and change the ip address (might be *) to 127.0.0.1. The lines will look like the following

NameVirtualHost 127.0.0.1:8080 <VirtualHost 127.0.0.1:8080>

Also edit the ports.conf file (sudo nano /etc/apache2/ports.conf) so that it will listen on 8080.

Listen 8080

Don't restart the server yet, you want to configure nginx first. Edit the default nginx config file (sudo nano /etc/nginx/sites-available/default) and find where it says

        location / {                root   /var/www/nginx-default;                index  index.html index.htm;         }

and replace it with

location / {     proxy_pass http://192.168.0.180:8080;     proxy_redirect off;     proxy_set_header Host $host;     proxy_set_header X-Real-IP $remote_addr;     proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;     client_max_body_size 10m;     client_body_buffer_size 128k;     proxy_connect_timeout 90;     proxy_send_timeout 90;     proxy_read_timeout 90;     proxy_buffer_size 4k;     proxy_buffers 4 32k;     proxy_busy_buffers_size 64k;     proxy_temp_file_write_size 64k;  } location /files/ {     root /var/www/myproject/;     expires max; }

/files/ is where I've stored all of my static files and /var/www/myproject/ is where my project lives and it contains the files directory.

Set static files to expire far in the future

expires max; will tell your users' browsers to cache the files from that directory for a long time. Only use that if you are use those files won't change. You can use expires 24h; if you aren't sure.

Configure gzip

Edit the nginx configuration to use gzip on all of your static files (sudo nano /etc/nginx/nginx.conf). Where it says gzip on; make sure it looks like the following:

    gzip  on;     gzip_comp_level 2;     gzip_proxied any;     gzip_types      text/plain text/html text/css application/x-javascript text/xml application/xml application/xml+rss text/javascript;

The servers should be ready to be restarted.

sudo /etc/init.d/apache2 reload sudo /etc/init.d/nginx reload

If you are having any problems I suggest reading through this guide and seeing if you have something set up differently.

Speedy Django Sites

Those three steps should speed up your server and allow for more simultaneous visitors. There is a lot more that can be done, but getting these three easy things out of the way first is a good start.

Django|compression and other best practices

Speed up Django with far-future expires, compression and other best practices — Greg Brown

Speed up Django with far-future expires, compression and other best practices

As a web developer with a shoddy rural internet connection, I'm always interested in speeding up my sites. One technique for doing this is far-future expires — i.e. telling the browser to cache media requests forever, then changing the uri when the media changes. In this article, I outline how to implement this and several other techniques in django.

Goals

  1. Reduce http requests for css and js files to a bare minimum.
  2. Add far-future-expires headers to all static content
  3. Gzip all css and js content
  4. Reduce css/js filesize by minification

The django-compress App

First up, I installed the django-compress app. I was about to build my own solution when I realised this one did exactly what I needed — gotta love the django community. Configuring is straightforward — the project wiki has articles on installation, configuration, and usage..

I copied the compress/ directory into my django/library/ folder (which is on the python path) and added "compress" to my INSTALLED_APPS.

Apache Configuration

Once I had django-compress up and running, I had achieved goal #1. To achieve #2 and #3 I needed to configure apache to send the right headers along with each request. To do this, I put the following directive in my httpd.conf file:

<DirectoryMatch /path-to-django-projects/([^/]+)/media>      Order allow,deny     Allow from all      # Insert mod_deflate filter     SetOutputFilter DEFLATE     # Netscape 4.x has some problems...     BrowserMatch ^Mozilla/4 gzip-only-text/html     # Netscape 4.06-4.08 have some more problems     BrowserMatch ^Mozilla/4\.0[678] no-gzip     # MSIE masquerades as Netscape, but it is fine     BrowserMatch \bMSIE !no-gzip !gzip-only-text/html     # Don't compress images     SetEnvIfNoCase Request_URI \     \.(?:gif|jpe?g|png)$ no-gzip dont-vary     # Make sure proxies don't deliver the wrong content     Header append Vary User-Agent env=!dont-vary      # MOD EXPIRES SETUP     ExpiresActive on     ExpiresByType text/javascript "access plus 10 year"     ExpiresByType application/x-javascript "access plus 10 year"     ExpiresByType text/css "access plus 10 years"     ExpiresByType image/png  "access plus 10 years"     ExpiresByType image/x-png  "access plus 10 years"     ExpiresByType image/gif  "access plus 10 years"     ExpiresByType image/jpeg  "access plus 10 years"     ExpiresByType image/pjpeg  "access plus 10 years"     ExpiresByType application/x-flash-swf  "access plus 10 years"     ExpiresByType application/x-shockwave-flash  "access plus 10 years"      # No etags as we're using far-future expires     FileETag none  </DirectoryMatch> 

Notes

  • <DirectoryMatch /path-to-django-projects/([^/]+)/media> is equivalent to writing <Directory /path-to-django-projects/site-name/media> for each site.
  • mod_deflate configuration directives from the Apache site.
  • Note that I'm sending far-future-expires headers for images and flash too — at this stage, that means I have to manually change the filenames whenever I change the content.

This means that for all my django sites' media directories:

  • static content (except images) is gzipped via mod_deflate
  • everything gets a header telling the browser to cache it for 10 years

Note you will need mod_deflate and mod_expires enabled in your apache config - if you have apache 2.2 it should just be a matter of copying the relevant files from apache2/mods-available/ to /apache2/mods-enabled/.

Minification

Step #4 was the trickiest of the lot, and many would argue that it's not really worth the trouble. Depending on how verbosely you comment your js and css, it may or may not be worthwhile for you — personally, I just thought I may as well go the whole hog. In the end, I probably only saved a few percent worth of bandwidth for my small content sites, but it'll be more significant with js-heavy web-apps.

For js minification, django-compress comes with jsmin built in. I've found this to be ideal for the job, and it is enabled by default.

For CSS, django-compress comes with CSSTidy — a CSS parser and optimiser — built in, in the form of csstidy_python. (You can also use a csstidy binary if you have one installed.) Personally, I find CSSTidy messes with my css, and more significantly, messes with that of my css framework of choice, 960.gs. I was after something that simply stripped whitespace, newlines and comments, without parsing the code. After scouring the web, I came across Slimmer — a lightweight pyhon app that did exactly what I needed. After installing it, I added the following file to the django-compress app's filters directory.

#compress/filters/slimmer_css/__init__.py  import slimmer from compress.filter_base import FilterBase  class SlimmerCSSFilter(FilterBase):     def filter_css(self, css):         return slimmer.css_slimmer(css) 

Then it was simply a matter of adding the following line to my settings.py file, as per the django-compress documentation:

COMPRESS_CSS_FILTERS = ('compress.filters.slimmer_css.SlimmerCSSFilter',) 

So my complete django-compress configuration in settings.py was as follows:

# compress app settings COMPRESS_CSS = {     'all': {         'source_filenames': (             'css/lib/reset.css',             'css/lib/text.css',             'css/lib/960.css',             'css/style.css',         ),         'output_filename': 'compress/c-?.css',         'extra_context': {             'media': 'screen,projection',         },     },      # other CSS groups goes here } COMPRESS_JS = {     'all': {         'source_filenames': ('js/lib/jquery.js', 'js/behaviour.js',),         'output_filename': 'compress/j-?.js',     }, }  COMPRESS = True COMPRESS_VERSION = True COMPRESS_CSS_FILTERS = ('compress.filters.slimmer_css.SlimmerCSSFilter',) 

Other best practices

I keep all my css within the <head> tags, and js at the bottom of the page — this is because the page browser needs to download all the css before it can start rendering the page, but doesn't need the js. It doesn't actually speed up the site, but it gives the impression of loading faster, and the user is unlikely to click on anything before the js has loaded anyway.

For a definitive guide, see Yahoo's performance rules. I also recommend Yahoo's YSlow, and if you are one of the 3 remaining web developers without it, Firebug.