tag:blogger.com,1999:blog-661358724048115072024-03-26T19:15:00.486-07:00My thoughts on computers, programming, ancient spell scrolls and other magic devices...Krzysztof Rosińskihttp://www.blogger.com/profile/10131798666482827067noreply@blogger.comBlogger61125tag:blogger.com,1999:blog-66135872404811507.post-30414502435971680992014-08-31T11:36:00.000-07:002014-08-31T11:36:46.487-07:00Possible I/O Errors while starting a new process from a daemon script (Popen)I recently had a problem with one of my document processing application crashing after trying to revive malfunctioned workers, with an:<br />
<br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">OSError: [Errno 5] Input/output error</span></span><br />
<br />
After identifying the problem I logged in to server via SSH, restarted the service with debug logging enabled and started processing a sequence of documents, that guaranteed that the workers will have to be restarted. I was really surprised to see that the service managed to cope with the problem and bring up all the workers. <br />
<br />
I verified that the only place that service could have crashed was starting the new worker process before the actual fork. I traced in python std libraries, that starting a new process eventually calls <span style="font-family: "Courier New",Courier,monospace;">Popen(self)</span>.<br />
<br />
Inspecting <span style="font-family: "Courier New", Courier, monospace;">multiprocessing/forking.py<b>:</b></span><br />
<br />
<br />
<pre class="brush:python">if sys.platform != 'win32':
# some not importent stuff
class Popen(object):
def __init__(self, process_obj):
sys.stdout.flush()
sys.stderr.flush()
self.returncode = None
self.pid = os.fork()
if self.pid == 0:
if 'random' in sys.modules:
import random
random.seed()
code = process_obj._bootstrap()
sys.stdout.flush()
sys.stderr.flush()
os._exit(code)
</pre>
<br />
So before <span style="font-family: "Courier New",Courier,monospace;">os.fork</span> is actually called, the script tries to flush standard output and error streams. So the IOError that was caused by our script trying to flush stderr/stdout to the /dev/tty device, which was unavailable after a period of time (dropping ssh session after starting the daemon). I investigated the script for any left behind <i>print</i> / logging <i>StreamHandlers</i>. After a long investigation it occurred that a 3-rd party library was rarely logging some errors using a StreamHandler...<br />
<br />
So the lesson learned is always verify that Your daemon scripts don't write stdout/stderr scripts or make sure the streams are redirect in the init script. Besides the fact that stderr won't provide any valuable information when You're off-line, You can easily run in to similar problems.<br />
<br />
KRKrzysztof Rosińskihttp://www.blogger.com/profile/10131798666482827067noreply@blogger.com0tag:blogger.com,1999:blog-66135872404811507.post-84289420590203308882014-01-25T20:14:00.003-08:002014-01-25T20:14:59.538-08:00Python list, set and dict comprehentions 2.7+Python supports list comprehension since v2.0. These expressions truly revolutionized python, making various functions much simpler and more readable. Let's see some basic LC in action:<br />
<br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">>>> [x for x in "test"]<br />['t', 'e', 's', 't']</span></span> <br />
<br />
Now often there is a need to generate a <b>set</b> or <b>dict</b> in a similar way, so I often see such code:<br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;"><br /></span></span>
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">>>> set([x for x in "test"])<br />set(['s', 'e', 't'])<br />>>> dict([(x,x) for x in "test"])<br />{'s': 's', 'e': 'e', 't': 't'}</span></span><br />
<br />
This is good: <br />
- it works!<br />
- it's more readable than implementing a for loop.<br />
<br />
But using python 2.7+ You can to it <b>better</b>! The latest python 2.7.X and 3.X support dict and set comprehensions - now this is pythonic! You can achieve the same results the following way:<br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;"><br /></span></span>
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">>>> {x for x in "test"}<br />set(['s', 'e', 't'])<br />>>> {x: x for x in "test"}<br />{'s': 's', 'e': 'e', 't': 't'}</span></span><br />
<br />
This is excellent!<br />
- it works!<br />
- it's more readable than creating a dict/set from a LC<br />
- <b>it's faster</b>!!!<br />
<br />
Simple performance comparison:<br />
<br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">>>> timeit.timeit('set([x for x in "test"])')<br />0.44252514839172363<br />>>> timeit.timeit('{x for x in "test"}')<br />0.37139105796813965<br /></span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">>>> timeit.timeit('dict([(x,x) for x in "test"])')<br />0.8899600505828857<br />>>> timeit.timeit('{x: x for x in "test"}')<br />0.3909759521484375</span></span><br />
<br />
Cheers!<br />
KRKrzysztof Rosińskihttp://www.blogger.com/profile/10131798666482827067noreply@blogger.com4tag:blogger.com,1999:blog-66135872404811507.post-69077575079157517132014-01-13T15:04:00.003-08:002014-01-13T15:04:48.658-08:00Preventing python from generating *pyc files on runtime<br />
Whenever You import a module, CPython compiles it to byte code, and saves it using the same path and filename, except for a *.pyc extension (valid for python 2.X). A python script converted to byte code <b>does not run any faster</b>, the only advantage is that <b>pyc files are loaded faster</b>. Although this is generally desired it may cause problems during development, when our application imports existing pyc files instead of compiling our freshly modified source files. Such problems should not occur to often, but when they do, we usually don't have a clue what's going on (I just fixed it, and it still crashes!?).<br />
<br />
Remove *pyc files!<br />
<br />
Of course You can create a script that performs a search-and-destroy on all *pyc files located in all of your projects subdirectories. This is cool, but preventing python from generating *pyc files (in dev) is even better.<br />
<br />
So there are basically three ways to achieve it in python 2.7:<br />
<br />
1. During script run, use the -B parameter<br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">python -B some_script.py</span></span><br />
<br />
2. Project level, insert this line on top your application / script (*py)<br />
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">import sys</span></span><br />
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">sys.dont_write_bytecode = True</span></span><br />
<br />
3. Environment level, set the following env. variable:<br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">export PYTHONDONTWRITEBYTECODE=1</span></span><br />
<br />
Have fun eradicating *pyc files in Your dev environments!<br />
<br />
Cheers!<br />
KR<br />
Krzysztof Rosińskihttp://www.blogger.com/profile/10131798666482827067noreply@blogger.com0tag:blogger.com,1999:blog-66135872404811507.post-29456537194219359412013-12-09T12:56:00.000-08:002013-12-09T13:55:29.963-08:00A few words on boolean casting/mapping in pythonSo I thought I'd write a few words about truth value testing in python. It's quite common to <i>if some_object </i>conditionals, and I believe that programmers are not always aware of what actually happens when such a line is being evaluated, and tend to perceive it only as <i>len(some_object) > 0</i> if it's a collection or <i>some_object is not None</i> in other cases. We may refer to the <a href="http://docs.python.org/2/reference/datamodel.html#object.__nonzero__">python docs</a> to verify it:<br />
<br />
<dl class="method">
<dt id="object.__nonzero__">
<tt class="descclassname">object.</tt><tt class="descname">__nonzero__</tt><big>(</big><i>self</i><big>)</big></dt>
<dd><div id="index-82">
Called to implement truth value testing and the built-in operation <tt class="docutils literal"><span class="pre">bool()</span></tt>;
should return <tt class="xref docutils literal"><span class="pre">False</span></tt> or <tt class="xref docutils literal"><span class="pre">True</span></tt>, or their integer equivalents <tt class="docutils literal"><span class="pre">0</span></tt> or
<tt class="docutils literal"><span class="pre">1</span></tt>. When this method is not defined, <a class="reference internal" href="http://docs.python.org/2/reference/datamodel.html#object.__len__" title="object.__len__"><tt class="xref py py-meth docutils literal"><span class="pre">__len__()</span></tt></a> is called, if it is
defined, and the object is considered true if its result is nonzero.
If a class defines neither <a class="reference internal" href="http://docs.python.org/2/reference/datamodel.html#object.__len__" title="object.__len__"><tt class="xref py py-meth docutils literal"><span class="pre">__len__()</span></tt></a> nor <a class="reference internal" href="http://docs.python.org/2/reference/datamodel.html#object.__nonzero__" title="object.__nonzero__"><tt class="xref py py-meth docutils literal"><span class="pre">__nonzero__()</span></tt></a>, all its
instances are considered true.</div>
<div id="index-82">
</div>
</dd></dl>
So basicly when we compose something like this:<br />
<br />
<pre class="brush:python">bool(some_obj)</pre>
<br />
Is evaluated like this:<br />
<br />
<pre class="brush:python">some_obj.__nonzero__() if hasattr(some_obj, '__nonzero__') else len(some_obj) <> 0 if hasattr(some_obj, '__len__') else True
</pre>
<br />
So basically conditionals that made with premeditation not from laziness, especially when using 3rd-party libraries/frameworks. For example, when using <a href="https://pypi.python.org/pypi/requests">python requests</a>,<br />
<br />
<pre class="brush:python">>>> import requests
>>> res = requests.get('https://google.pl/not_exists')
>>> res
<Response [404]>
>>> bool(res)
False
>>> res = requests.get('https://google.pl/')
>>> res
<Response [200]>
>>> bool(res)
True
</pre>
<br />
<br />
In case of python-requests http errors are not raised by default, instead if a status code of 4XX or 5XX is returned, the <i>__nonzero__</i> method returns False. In any case <i>res is not None</i> is always True in the above case.Krzysztof Rosińskihttp://www.blogger.com/profile/10131798666482827067noreply@blogger.com0tag:blogger.com,1999:blog-66135872404811507.post-21788572284173410102013-02-21T09:00:00.002-08:002013-02-21T09:00:57.186-08:00Python logging introspection<span style="font-family: Arial,Helvetica,sans-serif;"><span style="font-size: small;">How many times have you used a <i>print</i> instruction instead of <i>logger.debug </i>or <i>logger.info</i>? Well I used to do it frequently. The thing is, setting up a logger in an application that has many of its own, is problematic. There is a <a href="https://pypi.python.org/pypi/logging_tree">tool</a> however that may help you identify the right place for your logger (or identify the logger you want to use). </span></span><br />
<br />
<span style="font-family: "Courier New", Courier, monospace;">~ $ pip install logging_tree</span><br />
<br />
<span style="font-size: small;"><span style="font-family: Arial,Helvetica,sans-serif;">So what does this package do? In practice this is a logging introspection tool that recreates a tree structure of your current loggers (along with handlers and filters). This is very useful since you may immediately identify which logger you should use, or at least confirm that adding a new logger will be mandatory. </span></span><br />
<span style="font-size: small;"><span style="font-family: Arial,Helvetica,sans-serif;"><br /></span></span>
<span style="font-size: small;"><span style="font-family: Arial,Helvetica,sans-serif;">For example: if You type the following in a python terminal:</span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;"><br /></span></span>
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">>>> import logging_tree<br />>>> logging_tree.printout()<br /><--""<br /> Level WARNING</span></span><br />
<br />
<br />
<br />
Well this is quite obvious, no modules are loaded, thus no custom logger was registered. On the other hand let's look at a logging tree of a young django app:<br />
<br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">In [1]: import logging_tree <br /><br />In [2]: logging_tree.printout()</span></span><br />
<span style="font-size: x-small;"><br /><span style="font-family: "Courier New",Courier,monospace;"><--""<br /> Level WARNING<br /> |<br /> o<--"core"<br /> | Level INFO<br /> | Handler File '/tmp/wt.log'<br /> |<br /> o<--"django"<br /> | Handler Stream <open file '<stderr>', mode 'w' at 0x7f21e9890270><br /> | Filter <django.utils.log.RequireDebugTrue object at 0x1676350><br /> | |<br /> | o<--[django.db]<br /> | | |<br /> | | o<--"django.db.backends"<br /> | |<br /> | o<--"django.request"<br /> | Level ERROR<br /> | Handler <django.utils.log.AdminEmailHandler object at 0x1676790><br /> |<br /> o<--"nose"<br /> | |<br /> | o<--"nose.case"<br /> | |<br /> | o<--"nose.config"<br /> | |<br /> | o<--"nose.core"<br /> | |<br /> | o<--"nose.failure"<br /> | |<br /> | o<--"nose.importer"<br /> | |<br /> | o<--"nose.inspector"<br /> | |<br /> | o<--"nose.loader"<br /> | |<br /> | o<--"nose.plugins"<br /> | | |<br /> | | o<--"nose.plugins.attrib"<br /> | | |<br /> | | o<--"nose.plugins.capture"<br /> | | |<br /> | | o<--"nose.plugins.collect"<br /> | | |<br /> | | o<--"nose.plugins.cover"<br /> | | |<br /> | | o<--"nose.plugins.doctests"<br /> | | |<br /> | | o<--"nose.plugins.isolation"<br /> | | |<br /> | | o<--"nose.plugins.logcapture"<br /> | | |<br /> | | o<--"nose.plugins.manager"<br /> | | |<br /> | | o<--"nose.plugins.multiprocess"<br /> | | |<br /> | | o<--"nose.plugins.testid"<br /> | |<br /> | o<--"nose.proxy"<br /> | |<br /> | o<--"nose.result"<br /> | |<br /> | o<--"nose.selector"<br /> | |<br /> | o<--"nose.suite"<br /> |<br /> o<--[py]<br /> | |<br /> | o<--"py.warnings"<br /> | Handler Stream <open file '<stderr>', mode 'w' at 0x7f21e9890270><br /> | Filter <django.utils.log.RequireDebugTrue object at 0x1676350><br /> |<br /> o<--"south"<br /> Handler <south.logger.NullHandler object at 0x20c9350><br /> </span></span><br />
<br />
<span style="font-size: xx-small;"><span style="font-family: "Courier New",Courier,monospace;"><br /></span></span>It's much easier to read this tree output than getting familiar with your applications logging configuration along with the documentation of other packages that are using the logging module.<br />
<br />
Hope this saves You a lot of time.<br />
Cheers!<br />
<br />
KRKrzysztof Rosińskihttp://www.blogger.com/profile/10131798666482827067noreply@blogger.com0tag:blogger.com,1999:blog-66135872404811507.post-931870641822035082013-02-14T02:38:00.000-08:002013-02-14T02:38:17.423-08:00A non-production function decoratorAs most developers know, not every piece of code is meant to be run on a production server. Instead of using a lot of <i>"ifs" </i>here and there I suggest implementing a framework specific <i>"non_production"</i> decorator. A simple django-specific implementation could look like this:<br />
<br />
<pre class="brush:python">def non_production(func):
def is_production():
#django specific
from django.conf import settings
return getattr(settings, "PRODUCTION", False)
def wrapped(*args, **kwargs):
if is_production():
raise Exception("%s is not meant to be run on a production server" % \
func.__name__)
else:
return func(*args, **kwargs)
return wrapped
</pre>
<br />
Now all you have to do is to apply it to your dev/test only functions:<br />
<br />
<pre class="brush:python">@non_production
def test_something(a, b):
pass
</pre>
<br />
<br />
Cheers!<br />
KRKrzysztof Rosińskihttp://www.blogger.com/profile/10131798666482827067noreply@blogger.com0tag:blogger.com,1999:blog-66135872404811507.post-18517371264318554892013-01-31T10:36:00.002-08:002013-01-31T10:36:58.728-08:00Python code readability - PEP 8<span style="font-size: small;"><span style="font-family: Arial,Helvetica,sans-serif;">Everybody familiar with python should be aware there are a bunch of documents called PEPs (Python Enhancement Proposals). As the name states, this documents are intended to help improve python, not only by adding new features to the interpreter and enhancing standard libraries, they also give guidelines (proposals) about meta-programming. Many experienced and respectable members of the python community participate in the PEP life-cycle, thus the documents are really reliable. <br /><br />The thing I would like to talk about is <b>PEP 8</b>, also known as the <b>Style Guide for Python Code</b>.</span></span> As you may know, the proposed name convention differs a bit from other high level programming languages (like Java, whitch its known for <i>UlitimateLongAndDescriptiveClassName </i>naming convetion, along with other funny things like<i> evenLongerAndMoreSpohisticatedMethodNames</i>). Anyway it's very intuitive, 4 spaces instead of tab, underscore inside multi-word variables, etc. Sounds cool, you may even download a package that checks your type convention with reference to PEP 8:<br />
<b><span style="font-family: "Courier New",Courier,monospace;"><br /></span></b>
<b><span style="font-family: "Courier New",Courier,monospace;">~ $ pip install pep8</span></b><br />
<br />
Now you may check your perfectly compatible with PEP 8 source codes:<br />
<b><span style="font-family: "Courier New",Courier,monospace;"><br /></span></b>
<b><span style="font-family: "Courier New",Courier,monospace;">~ $ pep8 app/models.py</span></b><br />
<br />
and to your wonderment get a result similar to this:<br />
<br />
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">app/models.py:37:13: E128 continuation line under-indented for visual indent<br />app/models.py:36:58: E502 the backslash is redundant between brackets<br />app/models.py:42:1: W293 blank line contains whitespace<br />app/models.py:44:5: E303 too many blank lines (5)<br />app/models.py:63:80: E501 line too long (82 > 79 characters)<br />app/models.py:66:80: E501 line too long (95 > 79 characters)<br />app/models.py:67:80: E501 line too long (103 > 79 characters)<br />app/models.py:69:80: E501 line too long (82 > 79 characters)<br />app/models.py:70:80: E501 line too long (83 > 79 characters)<br />app/models.py:72:1: E302 expected 2 blank lines, found 1<br />app/models.py:80:1: W391 blank line at end of file</span></span><br />
<br />
A clean and tidy source file prints so many errors... well yes it does. In fact without additional IDE features it's nearly impossible to write PEP 8 valid code. If you're using vim, you can get a <a href="http://www.vim.org/scripts/script.php?script_id=2914">PEP 8 validation plug-in</a> that opens a quick-fix buffer with a list of PEP 8 incompatible statements. This is good for shaping your coding habits, but don't get to orthodox<b> - don't ever change an existing projects coding convention, j</b>ust keep to the current one. <br />
<br />
~Thus spoketh KR,<br />
<br />Krzysztof Rosińskihttp://www.blogger.com/profile/10131798666482827067noreply@blogger.com3tag:blogger.com,1999:blog-66135872404811507.post-85656834008496420972013-01-29T09:16:00.001-08:002013-01-29T09:16:29.058-08:00The forloop django tempalte variableDjango template for-loop iteration is usually executed on <i>QuerySets </i>(paginated or not). For front-end purposes it's sometimes good to provide row numbers (especially if the data is supposed to be presented in a specific order). A first non-django solution that comes my mind is something like this:<br />
<br />
<br />
<pre class="brush:python">
def test_view(request):<br />
ctx = {}<br />
objects = MyModel.objects.all()<br />
ctx['objects'] = zip(range(1,objects.count()+1), objects)<br />
return render_to_response("app/test.html", ctx)<br />
</pre>
<br />
<br />
And for the template:<br />
<br />
<pre class="brush:html">
<table><br />
{% for el in objects %}<br />
<tr><br />
<td>el.0</td><br />
<td>el.1.some_field</td><br />
{% comment %} Other fields.... {% endcomment %}<br />
</tr><br />
{% endfor %}<br />
</table><br />
</pre>
<br />
This is a great python solution, but <b>it's not a django solution</b>. First of all, by using <i>zip </i>function we select the records from the database, adding pagination now would be a bit complicated (using the <i>PaginationMiddleware </i>won't be effecient since it works well only on lazy <i>QuerySets</i>). Another Disadvantage is the need of addressing the counter and data by index. Of course we could make the counter an attribute of the data objects... but it's still not THE solution. <br />
<br />
Django provides a better way of solving this problem. Inside each for-loop you may access a <i>forloop</i> template variable. According to the <a href="https://docs.djangoproject.com/en/dev/ref/templates/builtins/">django docs</a> the following attributes are available:<br />
<br />
<table class="docutils"><tbody valign="top">
<tr><td><tt class="docutils literal"><span class="pre">forloop.counter</span></tt></td>
<td>The current iteration of the loop (1-indexed)</td>
</tr>
<tr><td><tt class="docutils literal"><span class="pre">forloop.counter0</span></tt></td>
<td>The current iteration of the loop (0-indexed)</td>
</tr>
<tr><td><tt class="docutils literal"><span class="pre">forloop.revcounter</span></tt></td>
<td>The number of iterations from the end of the
loop (1-indexed)</td>
</tr>
<tr><td><tt class="docutils literal"><span class="pre">forloop.revcounter0</span></tt></td>
<td>The number of iterations from the end of the
loop (0-indexed)</td>
</tr>
<tr><td><tt class="docutils literal"><span class="pre">forloop.first</span></tt></td>
<td>True if this is the first time through the loop</td>
</tr>
<tr><td><tt class="docutils literal"><span class="pre">forloop.last</span></tt></td>
<td>True if this is the last time through the loop</td>
</tr>
<tr><td><tt class="docutils literal"><span class="pre">forloop.parentloop</span></tt></td>
<td>For nested loops, this is the loop "above" the
current one</td><td></td><td></td></tr>
</tbody></table>
<br />
Now the template could look like this:<br />
<br />
<br />
<pre class="brush:html">
<table><br />
{% for el in objects %}<br />
<tr><br />
<td>{{forloop.counter}}</td><br />
<td>{{el.some_field}}</td><br />
{% comment %} Other fields.... {% endcomment %}<br />
</tr><br />
{% endfor %}<br />
</table><br />
</pre>
<br />
This is more elegant and practical. It's also easy to combine it with the <i>PaginationMiddleware</i>. Ale you need to do is add <i>forloop.counter</i> with each page start index.<br />
<br />
Cheers!<br />
KR Krzysztof Rosińskihttp://www.blogger.com/profile/10131798666482827067noreply@blogger.com0tag:blogger.com,1999:blog-66135872404811507.post-53699133803583337262013-01-26T08:38:00.001-08:002013-01-26T08:38:38.120-08:00Setting up a development tmux sessionLets face it, it takes some time to setup a development session. Besides the IDE there are usually lots of other scripts and tools that need to be run, and monitored throughout the development process. Usually you need to run each script/tool in a separate terminal. This is quite inconvenient since multiple tools could be aggregated (you usually do not need a full-screen version of <i>htop</i> running, same goes for logfiles). This problem may be solved using a terminal multiplexer, in our case <i><a href="http://www.openbsd.org/cgi-bin/man.cgi?query=tmux&sektion=1">tmux</a></i>. If you're missing it, installing it is a must:<br />
<br />
<span style="font-family: "Courier New", Courier, monospace;">~ $ sudo apt-get install tmux</span><br />
<br />
<span style="font-size: small;"><span style="font-family: Arial, Helvetica, sans-serif;">Using <i>tmux</i> is a </span></span>enables organizing your dev session in a better/tidier manner. The greatest advantage of using <i>tmux</i> over <i>screen</i> or a tabbed <i>gnome-terminal</i> is the possibility to splitting a pane (both horizontally and vertically), this eventually enables to setup your scrips/tools in any way you can imagine. Term 'eventually' was not used accidentally, eventually because it usually takes some typing to achieve the intended results. We programmers are lazy and like automating things, so why not create a script that sets up <i>tmux</i> with our predefined panes. This can be achieved by using <i><a href="https://github.com/aziz/tmuxinator">tmuxinator</a></i>. You can install it using <i>gem</i>.<br />
<span style="font-family: "Courier New",Courier,monospace;"><br /></span>
<span style="font-family: "Courier New",Courier,monospace;">~ $ sudo gem install tmuxinator</span><br />
<br />
Now we can create a new session definition:<br />
<br />
<span style="font-family: "Courier New", Courier, monospace;">~ $ tmuxinator new fxbot</span><br />
<br />
<span style="font-family: "Courier New", Courier, monospace;"><span style="font-family: Arial, Helvetica, sans-serif;">And type in some basic instructions to <span style="font-family: "Courier New",Courier,monospace;">~/.tmuxinator/fxbot.yml<span style="font-family: Arial, Helvetica, sans-serif;">:</span></span></span></span><br />
<br />
<span style="font-family: "Courier New", Courier, monospace;"><span style="font-family: Arial, Helvetica, sans-serif;"><span style="font-family: "Courier New",Courier,monospace;"><span style="font-family: Arial, Helvetica, sans-serif;"><span style="font-family: "Courier New",Courier,monospace;"># ~/.tmuxinator/fxbot.yml<br /><br />project_name: FXBot <br />project_root: ~/prj/forex/forex_bot/<br />tabs:<br /> - editor: vim .<br /> - console:<br /> layout: main-vertical<br /> panes:<br /> - #bash <br /> - ipython<br /> - stats:<br /> layout: main-vertical<br /> panes:<br /> - htop<br /> - tail -f logs.txt</span></span></span></span></span><br />
<span style="font-family: "Courier New", Courier, monospace;"><span style="font-family: Arial, Helvetica, sans-serif;"><span style="font-family: "Courier New",Courier,monospace;"><span style="font-family: Arial, Helvetica, sans-serif;"><br /></span> </span></span></span><br />
<span style="font-family: "Courier New", Courier, monospace;"><span style="font-family: Arial, Helvetica, sans-serif;"><span style="font-family: "Courier New",Courier,monospace;"><span style="font-family: Arial, Helvetica, sans-serif;">Now when you run:</span></span></span></span><br />
<span style="font-family: "Courier New", Courier, monospace;"><span style="font-family: Arial, Helvetica, sans-serif;"><span style="font-family: "Courier New",Courier,monospace;"><span style="font-family: Arial, Helvetica, sans-serif;"><br /></span> </span></span></span><br />
<span style="font-family: "Courier New", Courier, monospace;"><span style="font-family: Arial, Helvetica, sans-serif;"><span style="font-family: "Courier New",Courier,monospace;"><span style="font-family: Arial, Helvetica, sans-serif;"><span style="font-family: "Courier New",Courier,monospace;">~ $ tmuxinator start fxbot</span></span></span></span></span><br />
<span style="font-family: "Courier New", Courier, monospace;"><span style="font-family: Arial, Helvetica, sans-serif;"><span style="font-family: "Courier New",Courier,monospace;"><span style="font-family: Arial, Helvetica, sans-serif;"><br /></span> </span></span></span><br />
<span style="font-family: "Courier New", Courier, monospace;"><span style="font-family: Arial, Helvetica, sans-serif;"><span style="font-family: "Courier New",Courier,monospace;"><span style="font-family: Arial, Helvetica, sans-serif;">You will end up running <i>tmux</i> with three tabs, the <b>editor<i> </i></b>tab will contain a running instance of <i>vim</i>, the <b>console<i> </i></b>tab will be split vertically, the left pane will have a bash terminal, while the right will have <i>ipython </i>running. The final <b>stats</b> tab will be displaying <i>htop</i> and the last entries of a logfile. Well maybe its not a hard session to set up manually, but imagine setting up 5+ panes with split screens and various tasks running. </span></span></span></span><br />
<span style="font-family: "Courier New", Courier, monospace;"><span style="font-family: Arial, Helvetica, sans-serif;"><span style="font-family: "Courier New",Courier,monospace;"><span style="font-family: Arial, Helvetica, sans-serif;"><br /></span> </span></span></span><br />
<span style="font-family: "Courier New", Courier, monospace;"><span style="font-family: Arial, Helvetica, sans-serif;"><span style="font-family: "Courier New",Courier,monospace;"><span style="font-family: Arial, Helvetica, sans-serif;">This tool may also be configured to execute tasks before starting (like setting up the database server). More information is available on the projects site. </span></span></span></span><br />
<br />
<span style="font-family: "Courier New", Courier, monospace;"><span style="font-family: Arial, Helvetica, sans-serif;"><span style="font-family: "Courier New",Courier,monospace;"><span style="font-family: Arial, Helvetica, sans-serif;">Cheers!</span></span></span></span><br />
<span style="font-family: "Courier New", Courier, monospace;"><span style="font-family: Arial, Helvetica, sans-serif;"><span style="font-family: "Courier New",Courier,monospace;"><span style="font-family: Arial, Helvetica, sans-serif;">KR </span></span></span></span><br />
<span style="font-family: "Courier New", Courier, monospace;"><span style="font-family: Arial, Helvetica, sans-serif;"><span style="font-family: "Courier New",Courier,monospace;"><span style="font-family: Arial, Helvetica, sans-serif;"><br /></span> </span></span></span><br />
<span style="font-family: "Courier New", Courier, monospace;"><span style="font-family: Arial, Helvetica, sans-serif;"><span style="font-family: "Courier New",Courier,monospace;"><span style="font-family: Arial, Helvetica, sans-serif;">~KR</span> </span></span></span>Krzysztof Rosińskihttp://www.blogger.com/profile/10131798666482827067noreply@blogger.com0tag:blogger.com,1999:blog-66135872404811507.post-28234067566748602412013-01-19T03:58:00.001-08:002013-01-19T03:58:19.523-08:00Implementing advanced python decoratorsPython decorators may save a lot of time, through the ease of applying them with other functions/methods. In practice python decorators are something between the decorator pattern and macros. An example decorator that enables running a function only in a single process at a time is presented <a href="http://krosinski.blogspot.com/2012/04/preventing-python-script-from-running.html">here</a>. <br />
<br />
Today I'll present how to implement a decorator that enables a few ways to handle the embedded function. Such a mechanism is used in the <a href="http://docs.celeryproject.org/en/latest/index.html">celery project</a>.<br />
<br />
A quick celery task usage example:<br />
<br />
<pre class="brush:python">@celery.task
def add(x, y):
return x + y
res = add.delay(5,6) #asynchronous task
res = add(5,6) #synchronous task
</pre>
<br />
This is quite convenient, you may have such decorators applied to functions and still dynamically choose how the decorator should act in a specific context. An implementation of a similar decorator is presented below (with some test code):<br />
<br />
<pre class="brush:python">class SomeDecorator():
def __call__(self, func, some_flag=False):
def f(*args, **kwargs):
if some_flag:
print "Some flag is set"
print "Some decorator routines"
func(*args,**kwargs)
setattr(f, "with_flag", lambda *args, **kwargs: \
self.__call__(func, some_flag=True)(*args,**kwargs))
return f
@SomeDecorator()
def test(a, b):
print a, b
if __name__ == '__main__':
print "*****"
test(5, b = "test")
print "*****"
test.with_flag(7, b="test2")
</pre>
<br />
<br />
We have to be aware that after a decorator is used, each code referring to <i>test</i> will instead refer to <i>f</i><b> </b>(function object type)<i>. </i>To achieve our goal we have to provide <i>f</i> capabilities of performing different type of decorator activities. In this example code I have added a function attribute (line 11) which executes the default <i>__call__</i> method, and still passes arguments/keyword arguments to the embedded function.<br />
<br />
The standard output of running the presented code:<br />
<br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">*****<br />Some decorator routines<br />5 test<br />*****<br />Some flag is set<br />Some decorator routines<br />7 test2</span></span><br />
<br />
<span style="font-size: small;"><span style="font-family: Arial, Helvetica, sans-serif;">Feel<span style="font-size: small;"> free to use / modif<span style="font-size: small;">y / upgrade this code to </span></span></span></span>achieve desired results. <br />
<br />
Cheers!<br />
KRKrzysztof Rosińskihttp://www.blogger.com/profile/10131798666482827067noreply@blogger.com0tag:blogger.com,1999:blog-66135872404811507.post-22025397839678921262013-01-08T12:14:00.001-08:002013-01-08T12:16:12.346-08:00Incremental backups with rsyncThere are two types of computer users in the world: those who backup their data, and those who eventually will backup their data. Making regular backups consumes some time, but saves a lot of nerves and time(money) the primary data source crashes. Let's face it, backups are important.<br />
<br />
Program source codes usually do not make problems, there are distributed version control systems with remote repositories which are ideal for not only sharing but also backing up the data.<br />
<br />
Usually there are gigabytes of data that you will want to backup besides your source code. A good place for such backups is a remote storage or an external drive. If you'd like to automate the backup process as much as possible I suggest using <i>rsync</i>. It enables making incremental backups, which save your Internet bandwith / makes it faster to synchronize external drives. The following makes an incremental copy of some directories located in the home directory.<br />
<br />
<br />
<pre class="brush:javascript">declare -a SOURCE_DIRS=("a" "b" "img" )
BACKUP_DIR=/mnt/backup/
for source_dir in ${SOURCE_DIRS[@]}
do
echo "Current directory: $HOME/$source_dir"
rsync -a "$HOME/$source_dir" $BACKUP_DIR
done
sync
echo "Backup complete"
</pre>
<br />
This works exceptionally well. The <b>-a<i> </i></b>option stands for:<br />
<br />
<ul>
<li>-r, --recursive recurse into directories</li>
<li>-l, --links copy symlinks as symlinks</li>
<li>-p, --perms preserve permissions</li>
<li>-t, --times preserve modification times</li>
<li>-g, --group preserve group</li>
<li>-o, --owner preserve owner</li>
<li>--devices, preserve device files</li>
<li>--specials, preserve special files</li>
</ul>
More available at the <a href="http://linux.die.net/man/1/rsync">rsync manual</a>.<br />
<br />
Cheers!<br />
KRKrzysztof Rosińskihttp://www.blogger.com/profile/10131798666482827067noreply@blogger.com2tag:blogger.com,1999:blog-66135872404811507.post-21288213341936974852012-12-30T04:57:00.003-08:002012-12-30T04:57:48.883-08:00Laptop screen backlight brightness adjustment<span style="font-family: inherit;">Well I have to admit that not all of my Asus UL80V function keys work on Linux. Among them are the brightness adjustment buttons. Well it's not a problem to adjust the brightness on linux anyway, I remember there was a possibility to set it manually in:</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;"><span style="font-family: "Courier New",Courier,monospace;">/proc/acpi/video/VGA/LCD/brightness</span></span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;"><span style="font-family: Arial, Helvetica, sans-serif;">Sadly I failed to find the file on my current distribution (Mint 13 / Maya, based on Ubuntu 12). I made some research and searching and managed to locate another file responsible for the backlight brightness setting:</span></span><br />
<br />
<span style="font-family: inherit;"><span style="font-family: "Courier New",Courier,monospace;">/sys/devices/pci0000:00/0000:00:01.0/0000:01:00.0/backlight/acpi_video0/brightness </span></span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;">The brightness may be set the following way:</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: "Courier New",Courier,monospace;">sudo echo "15" > </span><span style="font-family: inherit;"><span style="font-family: "Courier New",Courier,monospace;">/sys/devices/pci0000:00/0000:00:01.0/0000:01:00.0/backlight/acpi_video0/brightness </span></span><br />
<br />
<span style="font-family: inherit;"><span style="font-family: "Courier New",Courier,monospace;"><span style="font-family: Arial, Helvetica, sans-serif;">This works, but is a little bit inconvenient. Now I need to find a way do bind this with the brightness increase/decrease buttons, which may be tricky since they don't appear in /dev/input/ events (or at least I haven't found a suitable event yet).</span></span></span><br />
<br />
<span style="font-family: inherit;"><span style="font-family: "Courier New",Courier,monospace;"><span style="font-family: Arial, Helvetica, sans-serif;">Cheers!</span></span></span><br />
<span style="font-family: inherit;"><span style="font-family: "Courier New",Courier,monospace;"><span style="font-family: Arial, Helvetica, sans-serif;">KR </span> </span></span>Krzysztof Rosińskihttp://www.blogger.com/profile/10131798666482827067noreply@blogger.com0tag:blogger.com,1999:blog-66135872404811507.post-59942857902488620452012-12-29T05:05:00.003-08:002012-12-29T05:05:35.729-08:00Generic django forms<a href="http://krosinski.blogspot.com/2012/12/dynamic-type-creation-in-python.html">My last post</a> presented a way of generating dynamic classes in python. Today I would like to present a way of applying this mechanism to Django forms, making them a bit generic. An example may be a login form, that may be used with an "accept regulations" checkbox or without it, depending on the context.<br />
<br />
<pre class="brush:python">class LoginForm(forms.Form):
username = forms.CharField(max_length=20, label="Username")
password = forms.CharField(widget=forms.\
PasswordInput(render_value=False),max_length=20, label="Password")
def clean(self):
#form validation/clean implementation
pass
</pre>
This a simplest login form you can get, all there is to do is to implement a login mechanism (or simply use it, it's available at the <i>django.contrib.auth</i> module). Both provided fields are required,<i> </i>so the <i>is_valid </i>method will generate an appropriate result. Now let's make a possibility to generate this form with an "accept regulations" which is also required.<br />
<br />
<br />
<pre class="brush:python">class LoginForm(forms.Form):
username = forms.CharField(max_length=20, label="Username")
password = forms.CharField(widget=forms.\
PasswordInput(render_value=False),max_length=20, label="Password")
def clean(self):
#form validation/clean implementation
pass
@classmethod
def with_accept_regulations(cls):
return type(
"%sWithAcceptRegulations" % cls.__name__,
(cls,),
{"accept_regulations" : forms.BooleanField(required=True, \
label="I accept the regulations")},
</pre>
Now we may easily obtain a login form with accept regulations using the <i>with_accept_regulations</i> class method. Example usage is presented below:<br />
<br />
<br />
<pre class="brush:python">if something:
CurrentLoginForm = LoginForm
else:
CurrentLoginForm = LoginForm.with_accept_regulations()
form = CurrentLoginForm(request.POST)
if form.is_valid():
#and os on...
pass
</pre>
This example is trivial, and a similar outcome may be achieved by using simpler mechanisms, however examples are suppose to be simple. This method may be applied to more demanding applications, a huge feature is the ability to chain class creation methods, for example you could try implementing the following:<br />
<br />
<i>LoginForm.with_accept_regulations().with_skin_selection(</i>), which would generate a class <i>LoginFormWithAcceptRegulationsWithSkinSelection. </i>I know this is starting to look like java, but the long names are meant to be descriptive :-)<br />
<br />
<br />
<br />
Have fun exploring new possibilities.<br />
<br />
Cheers!<br />
KR<br />
<br />
P.S.<br />
I was in a hurry, so the code may have some simple mistakes - beware :-) Krzysztof Rosińskihttp://www.blogger.com/profile/10131798666482827067noreply@blogger.com0tag:blogger.com,1999:blog-66135872404811507.post-30188066291143059142012-12-28T06:39:00.001-08:002012-12-28T06:39:54.372-08:00Dynamic type creation in pythonLong long time ago, in a galaxy far far away, before python 2.2+ was invented, classes and types were very different. Finally an <a href="http://www.python.org/download/releases/2.2/descrintro/">update</a> came and unified them. Nowadays those statements mean pretty much the same (python 2.7 / 3.X), so by stating "dynamic type creation" we actually mean "dynamic class creation" (not the other way round - this way it looks more OOP). There is a magic keyword <i>type, </i>which returns a type(class) of an object. <br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;"><br /></span></span>
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">>>> class Test(object):</span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">... pass</span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">... </span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">>>> inst = Test()</span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">>>> type(inst)</span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;"><class '__main__.Test'></span></span><br />
<br />
<br />
<br />
<br />
This seams logic, however the <i>type</i> function may be also used for dynamic class creation. We have to provide a new class name, a list of base classes and some additional attributes. For example we want to create a new class that extends <i>Test</i>.<br />
<br />
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">>>> Test2 = type("Test2", (Test,), {"my_func" : lambda self: "Hello"})</span></span><br />
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">>>> Test2</span></span><br />
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;"><class '__main__.Test2'></span></span><br />
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">>>> Test2.__dict__</span></span><br />
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">dict_proxy({'my_func': <function <lambda> at 0x12126e0>, '__module__': '__main__', '__doc__': None})</span></span><br />
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">>>> inst2 = Test2()</span></span><br />
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">>>> inst2.my_func()</span></span><br />
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">'Hello'</span></span><br />
<br />
This mechanism may not be so spectacular before you apply it with a sophisticated library/framework, tomorrow I'll present how to use it with django forms.<br />
<br />
Cheers!<br />
<br />
KR<br />
<br />
<br />Krzysztof Rosińskihttp://www.blogger.com/profile/10131798666482827067noreply@blogger.com0tag:blogger.com,1999:blog-66135872404811507.post-6446079498558042022012-12-17T12:33:00.000-08:002012-12-17T12:33:13.353-08:00Using table join with mysqldumpEverybody familiar with MySQL database knows <i><a href="http://linuxcommand.org/man_pages/mysqldump1.html">mysqldump</a></i>, the ultimate MySQL backup tool. This tool has more intereseting features, mainly it can be used to dump only specific tables or even selectively extract data from the database.<br />
<br />
So if You want to dump a single table, you should execute:<br />
<br />
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">mysqldump -u db_user -p db_name table_name > table_and_data.sql</span></span><br />
<br />
This is a default script, quite useful for recreating tables/making backups. But as it was mentioned before, we just want the data. Fortunately <i>mysqldump</i> comes with many options, <i>--no_create_info</i> for instance will drum only the data wrapped by an INSERT INTO statement. Another nice option: <i>--where </i>enables providing custom data filtering rules (typical for MySQL). Let's try a query that dumps specific rows from a table that is related to another table using a WHERE statement on the corresponding data model. <br />
<br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">mysqldump -u db_user -p db_name table_name --no_create_info \</span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">--lock-all-tables --where 'id in (select tn.id from table_name as tn \</span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">join related_table as rt on tn.related_table_id = rt.id \</span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">where rt.some_field = 1)' > data.sql</span></span><br />
<br />
The idea is to use a subselect to get <i>table_name </i>rows that have foreign keys to the <i>related_table </i>rows that meet the <i>rt.some_field = 1 </i>requirement. This works well, however if You access tables not specified in <i>mysqldump's</i> command params, You should lock them yourself. In this example I use <i>--lock-all-tables</i> which is an overkill (if You're using a transactional database engine, such as InnoDB, You can use <i>--single-transaction</i> instead - no table locking will be required).<br />
<br />
Cheers!<br />
~KR<br />
Krzysztof Rosińskihttp://www.blogger.com/profile/10131798666482827067noreply@blogger.com5tag:blogger.com,1999:blog-66135872404811507.post-70519908030494359072012-12-12T13:16:00.000-08:002012-12-12T13:16:45.483-08:00Mercurial disallow pushing multiple heads So why are multiple heads evil, and why do we want to stop <i>hg </i>from pushing them to another repo. Well, usually multiple heads are caused by failed merge attempts (avoiding merges). Pushing multiple heads in this situation will postpone the need of merging (it will be needed eventually), or make somebody else merge the heads. If a programmer intends to make a new head in the first place - he is likely to create a new branch.<br />
<br />
Another reason do want to push multiple head changesets are auto build/test frameworks. If the execute <i>hg incoming</i> and suddenly see 3 heads instead of one. Surely they'll fail to carry on their routine. <br />
<br />
I believe I've mentioned something about mercurial hooks and some of their applications. These hooks may be used to disallow pushing changesets with more than one head. We don't have to implement this hook by ourselves, a good implementation is available at <a href="http://hg.pythong.org/">hg.pythong.org</a>.<br />
<br />
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">mkdir ~/hooks && cd ~/hooks</span></span><br />
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">wget http://hg.python.org/hooks/archive/fc6022656a7b.tar.gz -O hooks.tar.gz</span></span><br />
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">tar zxf hooks.tar.gz<span style="font-size: x-small;"> <span style="font-size: x-small;">&& m<span style="font-size: x-small;">v hooks-fc6022656a7b/* .</span></span></span> </span></span><br />
<br />
Now we need to edit, the <i>~/.hgrc</i> file.<br />
<br />
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">[hooks]<br />pretxnchangegroup.checkheads = python:/home/kr/hooks/checkheads.py:hook</span></span><br />
<br />
<br />
Let's see how it works:<br />
<br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">~ $ ~/prj/python/bw $ hg push</span></span><br /><span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">pushing to /home/kr/repos/bw<br />searching for changes<br />abort: push creates new remote head 4ffea01f3220!<br />(did you forget to merge? use push -f to force)</span></span><br />
<br />
This was to be expected, by default mercurial required the -f switch to push a changeset containing multiple heads.<br />
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;"><br /></span></span>
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-size: x-small;">~ $ hg push -f<br />pushing to /home/kr/repos/bw<br />searching for changes<br />adding changesets<br />adding manifests<br />adding file changes<br />added 1 changesets with 1 changes to 1 files (+1 heads)<br />* You are trying to create new head(s) on 'default'!<br />* Please run "hg pull" and then merge at least two of:<br />* c96b37d67e47, 4ffea01f3220<br />transaction abort!<br />rollback completed<br />abort: pretxnchangegroup.checkheads hook failed</span></span><br />
<br />
This hook gives +10 to keeping your repo tidy!<br />
<br />
Cheers!<br />
KR<br />
<br />
P.S.<br />
There are some more hooks in the downloaded archive, feel free to check them out.<br />
<br />Krzysztof Rosińskihttp://www.blogger.com/profile/10131798666482827067noreply@blogger.com0tag:blogger.com,1999:blog-66135872404811507.post-55393956907929915562012-11-29T13:48:00.000-08:002012-11-29T13:48:33.722-08:00NumPy arrays vs regular arrays - common operationsNumPy is a set of scientific computing tools for python. I'm not going to talk about them, I mentioned NumPy, because it comes with an optimized array type <a href="http://docs.scipy.org/doc/numpy/reference/generated/numpy.array.html"><i>numpy.array</i></a>. Unlike casual python arrays, numpy.arrays work well with matrix/vector operators.<br />
<br />
Arrays and <i>numpy.arrays</i> do not share a common interface, but they both support iteration and support the <i>__setitem__ </i>and <i>__getitem__</i> methods. The first thing that differs is the initialization (obviously):<br />
<br />
<pre class="brush:python">>>> import numpy as np
>>> a = np.array([1,2,3,4,5])
>>> b = [1,2,3,4,5]
>>> a
array([1, 2, 3, 4, 5])
>>> b
[1, 2, 3, 4, 5]
</pre>
<br />
<br />
<br />
It's almost the same, you can iterate over both tables, set and get array elements. However when you want to add / remove elements you have to act differently - by using NumPy provided methods:<br />
<br />
<pre class="brush:python"> >>> np.delete(a, 2)
array([1, 2, 4, 5])
>>> del b[2]
>>> b
[1, 2, 4, 5]
</pre>
<br />
<br />
The prime difference here, is that casual array element deletion modifies the object, while np.delete creates a deep copy and than applies the effect. <i>Numpy.append </i>works in a similar maner, but instead it concatenates arrays (or adds elements).<br />
<br />
Now lets look at the <i>filter</i> function, these to instructions are equivalent:<br />
<br />
<pre class="brush:python">>>> filter(lambda x: x>2, b)
[3, 4, 5]
>>> a[a>2]
array([3, 4, 5])</pre>
<br />
<br />
And for the <i>map </i>function:<br />
<br />
<pre class="brush:python">>>> np_map = np.vectorize(lambda x: x**2)
>>> np_map(a)
array([ 1, 4, 9, 16, 25])
>>> map(lambda x: x**2, b)
[1, 4, 9, 16, 25]</pre>
<br />
<br />
Let's move on, and see where NumPy arrays have some more potential: vector operations. Let's suppose we want to make a sum of 2 vectors. Using regular python arrays you have to combine elements with the <i>zip / sum </i>functions, or just iterate and create sums on the fly. Example solution:<br />
<br />
<pre class="brush:python">>>> b = [1,2,3,4,5]
>>> d = [3,3,3,4,4]
>>> map(lambda x: sum(x), zip(b,d))
[4, 5, 6, 8, 9]</pre>
<br />
<br />
Now for the NumPy solution:<br />
<br />
<pre class="brush:python">>>> a = np.array([1,2,3,4,5])
>>> b = np.array([3,3,3,4,4])
>>> a + b
array([4, 5, 6, 8, 9])</pre>
<br />
<br />
And this works for all python binary operators, this is just awesome. When doing 'scientific' stuff, it's good to use appropriate data types. What's more, NumPy is very popular among other projects, so seeing <i>numpy</i>.<i>array</i>'s from time to time doesn't surprise nobody. I guess matrices and more complex operations will be covered some other day.<br />
<br />
Cheers!<br />
KR<br />
<br />
<br />Krzysztof Rosińskihttp://www.blogger.com/profile/10131798666482827067noreply@blogger.com0tag:blogger.com,1999:blog-66135872404811507.post-20479201500337531402012-11-27T11:10:00.001-08:002012-11-27T11:10:07.806-08:00Exporting MySQL queries into a CSV file.Every now and then I need to do some non-standard data processing things, which requires grabbing a bit of data from a database. Usually this "non-standard data processing things" are simple scripts, and enhancing them to get data directly from a database would kill their main advantages: simplicity and a fast setup time. <br />
<br />
CSV files are way better for this kind of processing. Since the data is located in a database, not a CSV file, we'll have to export it. Now there are two basic ways:<br />
<br />
a) The MySQL geek way.<br />
b) The bash/sh geek way.<br />
<br />
So let's start with the pure MySQL way. Usually when queries are made using the standard mysql console client, the retrieved data is formated into something that should reassemble a table... well maybe it does, but "broad" result tables aren't readable anyway.<br />
<br />
<pre class="brush:javascript">select * from some_table into outfile "/tmp/output.csv" \
fields terminated by ";" lines terminated by "\n";
</pre>
<br />
All we have to do is change the field and line terminator to match our needs. About the file, you should pass a path that is <b>writable for the mysql user</b>. If the file exists, or the MySQL server has no write permissions, this query will fail.<br />
<br />
Let's move on to the bash geek way - personally I don't fancy MySQL trciks, therefore I usually rely on bash. So here we go:<br />
<br />
<pre class="brush:javascript">echo "select * from some_table;" | \
mysql -u my_usr -p my_db | sed 's/\t/;/g' | \
tail -n +2 > /home/chriss/output.csv
</pre>
We execute a query, passing it via an input stream to the MySQL client. The client returns a tab separated list of fields (one per line), <i>sed </i>replaces those with semicolons. Next we dispose of the first line (using tail), since it contains column names. Finally we save the output as a file. Unlike in the MySQL way, you can write the file<b> </b>according to <b>your write permissions, not mysql's users</b>.<br />
<br />
Both approaches have some advantages and disadvantages, so their usage should be dependent on the context. <br />
<br />Krzysztof Rosińskihttp://www.blogger.com/profile/10131798666482827067noreply@blogger.com0tag:blogger.com,1999:blog-66135872404811507.post-75746928214815396332012-11-23T14:58:00.001-08:002012-11-23T14:58:35.011-08:00Django admin: handling relations with large tablesOne of the reasons, that Django became so popular, is that it provides a set of flexible administrative tools. You may setup your data management dashboard in relatively short time using some predefined classess.<br />
<br />
A typical object admin view consists of a <i>ModelForm</i> of an appropriate model, and some basic actions (save, add, delete). As you probably know, foreign keys are often represented as HTML select inputs. This works cool, when the related table is not vertically long, for example: electronic equipment brands. In such cases the combobox works well.<br />
<br />
Imagine a situation when, the foreign key points to a user record, a purchase order, or any other relation that does not have a limited number of rows. According to the previous example, the combo box should have thousands of options. This is a real performance killer, the ORM has to fetch the whole user/purchase order table in order to display a single object in an editable admin view. Don't be surprised if your query gets killed with the <i>"2006 - MySQL server has gone away"</i> status. <br />
<br />
There is a simple way to solve this: instead of selecting the whole table for presenting available options, we may present only the currently related object (its primary key). To achieve this, we must mark the foreign keys as <i>raw_id_fields</i>. Below sample usage:<br />
<br />
<br />
<pre class="brush:python">class PurchaseReportAdmin(admin.ModelAdmin):
#(...) other fields
raw_id_fields = ['purchase_order']
admin.site.register(PurchaseReport, PurchaseReportAdmin )
</pre>
<br />
<div>
<br />Yes, it's as simple as that. Keep in mind, that in many cases using raw id fields won't be necessary, but when it come to vertically huge tables - it's a good time/performance saving solution.<br />
<br />
Cheers!<br />
KR</div>
Krzysztof Rosińskihttp://www.blogger.com/profile/10131798666482827067noreply@blogger.com0tag:blogger.com,1999:blog-66135872404811507.post-85048979265435079982012-11-14T09:23:00.000-08:002012-11-14T09:23:59.724-08:00Selenium Firefox WebDriver and proxies.<a href="http://seleniumhq.org/">Selenium</a> is an functional web application testing system. It really helps developing test cases for apps that have a lot of logic implemented in the frontend layer. Selenium provides<i> WebDrivers</i> for most popular browsers, like IE, Chrome and last buy not least: Firefox. Of course Selenium has python bindings, which makes it even better. <br />
<br />
Suppose you need to use proxy servers for some of your scenarios (unique visitors). Since each test case may use many proxies, changing the global system proxy settings is a rather bad idea. Fortunately HTTP proxies may be configured in a web browser. If you inspect the Firefox <i>about:config</i> panel, you will notice many properties that configure how your browser really acts. We'll be interested in the <i>network.proxy.* </i>settings.<br />
<br />
<pre class="brush: python;">from selenium import webdriver
from selenium.common.exceptions import TimeoutException
try:
proxy = "192.168.1.3"
port = 8080
fp = webdriver.FirefoxProfile()
fp.set_preference('network.proxy.ssl_port', int(port))
fp.set_preference('network.proxy.ssl', proxy)
fp.set_preference('network.proxy.http_port', int(port))
fp.set_preference('network.proxy.http', proxy)
fp.set_preference('network.proxy.type', 1)
browser = webdriver.Firefox(firefox_profile=fp)
browser.set_page_load_timeout(15)
browser.get('http://192.168.1.1/test')
print browser.find_element_by_id('my_div').text
except TimeoutException as te:
print "timeout"
except Exception as ex:
print ex.message
finally:
browser.quit()
</pre>
<br />
First we need to configure our profile by setting the HTTP, and HTTPS proxies. Secondly we have to tell Firefox to start using them: <i>network.proxy.type -> </i>1. Finally we create an instance of Firefox with our proxy settings and execute out test routine.<br />
<br />
Have fun!<br />
<br />
~KR Krzysztof Rosińskihttp://www.blogger.com/profile/10131798666482827067noreply@blogger.com10tag:blogger.com,1999:blog-66135872404811507.post-40816057843752673482012-11-08T11:49:00.002-08:002012-11-08T11:49:32.826-08:00Python 2.7 forward compatibility with python 3.XAs many of You know, python is such a strange language, that releases are not backward compatible. Ok, it's really not that bad, the problem only occurs when migrating from python 2.X to 3.X. Features introduced in python 3 greatly increase the interpreter performance (such as decimal number processing, function annotations) and provide other useful mechanisms (ex. exception chaining). Some of this things were technically not possible to achieve in python 2.X, so thats where the conflicts come from.<br />
<br />
So if python 3 is that awesome, why do people still use python 2.7 (or older versions)? Most existing python frameworks and libraries do not support python 3, they are being ported but it will take some time for the whole python community to move on to the currently proposed version. <br />
<br />
So if you plan migrating to python 3 in the future, you best use the <i>__future</i>__ module. This module provides some basic forward compatibility for older versions of python. Two major differences that may crash your code are the <i>print</i> statement and literals encoding. You may observe <i>__future__</i> below, first the print function: <br />
<br />
<span style="font-family: "Courier New",Courier,monospace;">>>> print "test"<br />test<br />>>> from __future__ import print_function<br />>>> print "test"<br /> File "<stdin>", line 1<br /> print "test"<br /> ^<br />SyntaxError: invalid syntax<br />>>> print("test")<br />test<br />>>> </span><br /><br />
After importing <i>print_function </i>you are obligated to use <i>print </i>as a function. And now for unicode literals:<br />
<span style="font-family: "Courier New",Courier,monospace;"><br /></span>
<span style="font-family: "Courier New",Courier,monospace;">>>> isinstance("test", str)<br />True<br />>>> isinstance("test", unicode)<br />False<br />>>> from __future__ import unicode_literals<br />>>> isinstance("test", str)<br />False<br />>>> isinstance("test", unicode)<br />True<br />>>> "test"<br />u'test'<br />>>> </span><br />
<br />
All string are now by default encoded as unicode... personally I'm happy with it, since I really hated the "Ascii codec can't decode..." errors.<br />
<br />
<br />Krzysztof Rosińskihttp://www.blogger.com/profile/10131798666482827067noreply@blogger.com0tag:blogger.com,1999:blog-66135872404811507.post-9920897117640003782012-10-30T15:10:00.002-07:002012-10-30T15:10:56.621-07:00Evaluating minimum/maximum regex match length.The application of regular expressions in data validations is a standard nowadays. A good regular expression in the right context may save the day, I believe no one can deny it. However, sometimes you (or other people using your software) need some more feedback if the provided data fails to match the test pattern. A typical situation may look like this:<br />
<br />
<pre class="brush:python">def test_id(user_id):
import re
regex = r"(secret\s?)?agent\s([0-9]{3}|james bond)"
if not re.match(regex, user_id):
print "Incorrect user ID: %s" % user_id
else:
print "OK: %s" % user_id
test_id('agent james bond')
test_id('secret agent 007')
test_id('agent jack black')
test_id('agent j')
</pre>
<br />
Running this code results in the following output:<br />
<br />
<span style="font-family: "Courier New",Courier,monospace;">OK: agent james bond<br />OK: secret agent 007<br />Incorrect user ID: agent jack black<br />Incorrect user ID: agent j</span><br />
<br />
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-family: Arial,Helvetica,sans-serif;">So what if we want to give a hint, about the minimum and maximum length? In many cases guessing/</span></span><span style="font-family: "Courier New",Courier,monospace;"><span style="font-family: Arial,Helvetica,sans-serif;"><span style="font-family: "Courier New",Courier,monospace;"><span style="font-family: Arial,Helvetica,sans-serif;">manually </span></span>evaluating the potential match size should be easy. However, if You plan using some more sophisticated regular expressions, You should not rely on Your intuition - but on calculations:</span></span><br />
<br />
<br />
<pre class="brush:python">def test_id(user_id):
import re
import sre_parse
regex = r"(secret\s?)?agent\s([0-9]{3}|james bond)"
match_range = sre_parse.parse(regex).getwidth()
if len(user_id) < match_range[0] or \
len(user_id) > match_range[1]:
print "User ID %s should be between %s and %s characters" % \
(user_id, match_range[0], match_range[1])
elif not re.match(regex, user_id):
print "Incorrect user ID: %s" % user_id
else:
print "OK: %s" % user_id
test_id('agent james bond')
test_id('secret agent 007')
test_id('agent jack black')
test_id('agent j')
</pre>
<br />
<span style="font-family: Arial,Helvetica,sans-serif;">This time the output should look like this:</span><br />
<br />
<span style="font-family: "Courier New",Courier,monospace;">OK: agent james bond<br />OK: secret agent 007<br />Incorrect user ID: agent jack black<br />User ID agent j should be between 9 and 23 characters</span><br />
<span style="font-family: "Courier New",Courier,monospace;"></span><br />
<span style="font-family: Arial, Helvetica, sans-serif;">It certainly is more descriptive.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;">Cheers!</span><br />
<br />
~KR<br />
<span style="font-family: "Courier New",Courier,monospace;"></span><br />
<span style="font-family: "Courier New",Courier,monospace;"><br /></span>
<br />Krzysztof Rosińskihttp://www.blogger.com/profile/10131798666482827067noreply@blogger.com0tag:blogger.com,1999:blog-66135872404811507.post-63194555938331816012012-10-27T07:25:00.000-07:002012-10-27T07:25:22.421-07:00Determine Django form field clean orderDjango provides some excellent out-of-the-box form validation/processing mechanisms. These include casual forms, model based forms, and formsets. One of their features is the ease of implementing data validation procedures. According to the <a href="https://docs.djangoproject.com/en/dev/ref/forms/validation/">documentation</a>, the validation process includes executing the following methods in the presented order:<br />
<br />
<ol>
<li><span style="font-family: Arial,Helvetica,sans-serif;"><span class="docutils literal"><span class="pre">to_python (field)</span></span></span> </li>
<li><span style="font-family: Arial,Helvetica,sans-serif;"><span class="docutils literal"><span class="pre">validate (field)</span></span></span></li>
<li><span style="font-family: Arial,Helvetica,sans-serif;"><span class="docutils literal"><span class="pre">run_validators (field)</span></span></span></li>
<li><span style="font-family: Arial,Helvetica,sans-serif;"><span class="docutils literal"><span class="pre">clean (field)</span></span></span></li>
<li><span style="font-family: Arial,Helvetica,sans-serif;"><b><span class="docutils literal"><span class="pre">clean_<fieldname> (form)</span></span></b></span></li>
<li><span style="font-family: Arial,Helvetica,sans-serif;"><span class="docutils literal"><span class="pre"><b>clean (form)</b> </span></span></span></li>
</ol>
Since there is usually no need to extend form fields, let's focus on the form-side validation. Let's suppose you have the following forms:<br />
<br />
<pre class="brush:python">class MyBaseForm(forms.Form):
base_field1 = forms.CharField()
base_field2 = forms.CharField()
def clean_base_field1(self):
#(...)
return self.cleaned_data['base_field1']
def clean_base_field2(self):
#(...)
return self.cleaned_data['base_field2']
class MyForm(MyBaseForm):
field1 = forms.CharField()
def clean_field1(self):
#(...)
return self.cleaned_data['field1']
</pre>
<br />
All methods implemented in this example refer to step 5: <i>clean_<fieldname></i>. So what would be the execution order if we try validating <i>MyForm</i>? Django manual states:<br />
<span style="font-family: Times,"Times New Roman",serif;"><i><br /></i></span>
<span style="font-family: Times,"Times New Roman",serif;"><i>"These methods are run in the order given above, one field at a time. That is, for each field in the form (in the order they are declared in the form definition), the Field.clean() method (or its override) is run, then clean_<fieldname>(). Finally, once those two methods are run for every field, the Form.clean() method, or its override, is executed."</i></span><br />
<br />
According to this statement, the expected order would be: <i>clean_base_field1</i>, <i>clean_base_field2</i>, <i>field1</i>. What if we don't like this order, should we rearrange the form definition? No such thing! There is a way change this order in a more elegant way. We may use <i>fields.keyOrder </i>to achieve it:<br />
<br />
<pre class="brush:python">class MyForm(MyBaseForm):
field1 = forms.CharField()
def clean_field1(self):
#(...)
return self.cleaned_data['field1']
def __init__(self, *args, **kwargs):
super(MyForm, self).__init__(*args, **kwargs)
self.fields.keyOrder = ['base_field1', 'field1', 'base_field2']
</pre>
<br />
You don't need to extend a form to use this. You may also specify a partial order (if there are to many fields to be named explicitly) by putting only a few field names in the array and extending it by a list comprehension generated from the remaining <i>self.fields.keys()</i><i>. </i><br />
<br />
Cheers!<i> </i> <br />
<br />
~KR<br />
<br />
<br />Krzysztof Rosińskihttp://www.blogger.com/profile/10131798666482827067noreply@blogger.com1tag:blogger.com,1999:blog-66135872404811507.post-36143361842687602692012-10-23T13:48:00.002-07:002012-10-23T13:48:18.926-07:00Hard links vs. symbolic linksEverybody uses links in his everyday virtual life. I don't think anyone could imagine navigating between websites without using hyper-refs, but let's move to links that refer to our file-system. If You had some experience with MS Windows before, You're probably familiar with symbolic links, aka shortcuts.<br />
<br />
In general symbolic links may be interpreted as pointers directed to our files logical layer. In a larger scope this may look like a pointer to the pointer of physical data. If this still looks confusing, have a look at an example:<br />
<br />
<span style="font-family: "Courier New",Courier,monospace;">~ $ echo "Test file" > f1.txt</span><br />
<span style="font-family: "Courier New",Courier,monospace;">~ $ ln -s f1.tx f2.txt </span><br />
<br />
<span style="font-family: Arial, Helvetica, sans-serif;">This may be visualized the following way ("Test file" is the physical data here):</span><br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi-iQr0QnGaJ5clQzDFqCQIiSqdspeKwEGdQ2VCRHXsGj-5Vd2u_aBzeUWaA2theAEmXE55_NJBqdfIrwlmE643r2Vzauqe005yhfeY9HT4hVlkaJDIW7I72UKkrZPvtBr_fJD2pz5IE4c/s1600/inode1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img alt="symbolic link" border="0" height="160" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi-iQr0QnGaJ5clQzDFqCQIiSqdspeKwEGdQ2VCRHXsGj-5Vd2u_aBzeUWaA2theAEmXE55_NJBqdfIrwlmE643r2Vzauqe005yhfeY9HT4hVlkaJDIW7I72UKkrZPvtBr_fJD2pz5IE4c/s320/inode1.png" width="320" /></a></div>
Now you can access the physical data using both f1.txt and f2.txt. However, if You delete f1.txt, the physical data will be lost (no label will point to it). A different situation occurs when You use hard links instead. Each time You create a file, a label is hard linked to it. In the previous example the hard link was created by executing:<br />
<br />
<span style="font-family: "Courier New",Courier,monospace;">~ $ echo "Test file" > f1.txt</span> <br />
<br />
By default each chunk of physical data has only 1 hard link attached, but more may be attached. For example:<br />
<br />
<span style="font-family: "Courier New",Courier,monospace;">~ $ ln f1.txt f2.txt </span><br />
<br />
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-family: Arial, Helvetica, sans-serif;">Will create a hard link with label f2.txt to the physical data of f1.txt, let's visualize it:</span></span><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgVemJ6A-UdNhP4LMq8esfi0OHOz8EEignLh6SU_1yRwrPSb07AiWF1DfW6sjFAr9TvI2D4s0QXaE1BNUHw7JSGzqVNCKLlZJR7JKKvIn-M-iKKpCj3hpswca-ev9fk9m6BWpDgBYgooac/s1600/inode2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img alt="hard link" border="0" height="160" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgVemJ6A-UdNhP4LMq8esfi0OHOz8EEignLh6SU_1yRwrPSb07AiWF1DfW6sjFAr9TvI2D4s0QXaE1BNUHw7JSGzqVNCKLlZJR7JKKvIn-M-iKKpCj3hpswca-ev9fk9m6BWpDgBYgooac/s320/inode2.png" width="320" /></a></div>
You can access the physical data both via f1.txt and f2.txt. Unlike symbolic links, each of the hard links works even if the other stops to exist. In order to delete the physical data you need to unlink all hard links that point to it (rm f1.txt alone will not do...). <br />
<br />
To sum up, symbolic links are cool because they operate on the logical layer, and thus are not limited to a single file system. Furthermore symbolic links may point to directories which is an important feature.<br />
<br />
Hard links also have some features that their symbolic cousins have not. Hard links are always bound the the physical source of data, thus making them move/rename proof (symbolic links are not updated if you move/rename a corresponding hard link).<br />
<br />
Hope this overview helps you to choose a right link for each situation.<br />
<br />
Cheers!<br />
<br />
~KR<br />
Krzysztof Rosińskihttp://www.blogger.com/profile/10131798666482827067noreply@blogger.com0tag:blogger.com,1999:blog-66135872404811507.post-73713769835365966622012-10-15T14:31:00.004-07:002013-12-04T09:32:42.978-08:00Renaming Mercurial branchesI believe there is no need to present Mercurial (since You got here, You should be familiar with it anyway). I'd like to present a way of renaming / replacing branches. By default, without using extensions, it's impossible ("branches are permanent and global...")... but there are other ways to deal with it. Let us suppose we want to rename branch <i>A</i> to <i>B</i>. We can achieve it the following way:<br />
<br />
<span style="font-family: "Courier New",Courier,monospace;">hg update A</span><br />
<span style="font-family: "Courier New",Courier,monospace;">last_head=$(hg id -i)</span><br />
<span style="font-family: "Courier New",Courier,monospace;">hg ci -m "close A" --close-branch</span><br />
<span style="font-family: "Courier New",Courier,monospace;">hg update $last_head</span><br />
<span style="font-family: "Courier New",Courier,monospace;"><br /></span>
<span style="font-family: "Courier New",Courier,monospace;">hg branch B</span><br />
<span style="font-family: "Courier New",Courier,monospace;">hg ci -m "branche renamed to B"</span><br />
<br />
This is it, now our branch is named <i>B. </i>In practice we just closed branched <i>A </i>and created a new branch <i>B </i>from the last commit. This activity may be visualized the following way: <br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj-2yTIF0Z4nzt_F7bgpSqRuZ9XhxDmuDy8oAq2kVuessLRSZnr57fVtxpiyEkLxhK-Fd68Ck-fcfQiEg5NBIPLybVl3jG46q54DjeZImbp4SdkT-WQVJKpSNooZycGrnCQnvv48lfTync/s1600/hg_rename.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img alt="rename branch" border="0" height="96" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj-2yTIF0Z4nzt_F7bgpSqRuZ9XhxDmuDy8oAq2kVuessLRSZnr57fVtxpiyEkLxhK-Fd68Ck-fcfQiEg5NBIPLybVl3jG46q54DjeZImbp4SdkT-WQVJKpSNooZycGrnCQnvv48lfTync/s320/hg_rename.png" width="320" /></a></div>
Replacing an existing branch with another is a bit more tricky, here's what you have to do:<br />
<br />
<span style="font-family: "Courier New",Courier,monospace;">hg update A</span><span style="font-family: "Courier New",Courier,monospace;"> </span><br />
<span style="font-family: "Courier New",Courier,monospace;">hg ci -m "close A" --close-branch </span><br />
<span style="font-family: "Courier New",Courier,monospace;">hg update B</span><br />
<span style="font-family: "Courier New",Courier,monospace;">hg branch A -f #force, branch A exists</span><br />
<span style="font-family: "Courier New",Courier,monospace;">hg ci "rename to A"</span><br />
<span style="font-family: "Courier New",Courier,monospace;">#optional close branch B</span><br />
<br />
<span style="font-family: Arial,Helvetica,sans-serif;">The general idea may be presented the following way:</span><br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgq0tJ2uDO1Qh7Vn_x4fIeR2nSxoEbGIQCc5Bw8EY2Wey2MdGhgYrpZoSOmYgSSxIfY7HClWIQIRdvakq3WMIq-qq4mNNRB4aKtXNuWfhf3972sv3NQoif_Ca2asDcd7U284pTAQozMU7o/s1600/hg_replace.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img alt="replace branch" border="0" height="96" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgq0tJ2uDO1Qh7Vn_x4fIeR2nSxoEbGIQCc5Bw8EY2Wey2MdGhgYrpZoSOmYgSSxIfY7HClWIQIRdvakq3WMIq-qq4mNNRB4aKtXNuWfhf3972sv3NQoif_Ca2asDcd7U284pTAQozMU7o/s320/hg_replace.png" width="320" /></a></div>
<br />
<br />
<span style="font-size: small;"><span style="font-family: Arial, Helvetica, sans-serif;">In order to create a branch that previously existed we have to use the <i>force</i> </span></span>switch (<i>hg branch</i>). Nothing should go wrong if the previous head of branch <i>A </i>was closed, else You'll just end up creating another head.<br />
<br />
<br />
Experimenting with <i>hg</i> is cool, just remember - before You try anything experimental, commit your work! This may save You a lot of nerves.<br />
<br />
Cheers!<br />
<br />
~KR Krzysztof Rosińskihttp://www.blogger.com/profile/10131798666482827067noreply@blogger.com2