But what is the meaning of Kanban applied to software engineering?
The Kanban system was developed in Toyota more than half a century ago, and is one of the key points of lean manufacturing; its translation in software engineering, however, is not as straightforward and actually I am of the opinion the name is a bit misleading.

In our case, Kanban focuses on workflows, and their optimization.
As it focuses on process improvement, Kanban may be applied to all kind of development processes; the only requirements are a whiteboard, and a good configuration management system.

Analyzing the process, workflow and policies are made explicit.
Then, a limit is set to the work in progress – it means that each phase of the workflow is not allowed to proceed (and may even stop) until the next step is cleared.
Finally, there’s the optimization, based on actual, measured data. Optimization is continuous process, and usually affects workflow itself.

The main advantage of the system lays in its explicitness.
When the process is explicit, everybody can improve it, at any moment: any proposal may be confronted with the hard data, and changes are easier. On the other hand, issues are immediately spotted – just because they will impact on the flow, and there is no way to hide it.
As the process gets streamlined, it becomes more efficient and predictable than before.

However, the plugin ecosystem is healthy, and I was then able to find a good replacement in Amateras HTML Editor Plugin. I quite like it, and even if I have to edit the markup code, I can still see the preview, so it is not so bad after all. And the other tools (like palette, and especially the css editor) are well designed, and helps a lot. Yes, that’s not Dreamweaver, but for the moment I can live with that.

Another option, finally, is Aptana Studio. I am using it right now, and it’s better than Amateras. However, it is more invasive, and it has a slight “adware” attitude, because it has full of links for the “extended” version of the plugin, that requires some form of subscription. Nevertheless, it allows for a far better preview of html files, and if you do not care for those links, or (like me) avoid using the Aptana perspective, then it is quite OK: it still forces you to work on source level, but it has a good amount of helpers and the preview works enough to be useful.

The Zen of Python, by Tim Peters

Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren’t special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one– and preferably only one –obvious way to do it.
Although that way may not be obvious at first unless you’re Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it’s a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea — let’s do more of those!

As a matter of facts, this trick is pretty old – at least, as old as Windows XP.
But it is the kind of discover that amuses me like a little boy :-)

I bought it because I was interested in software contracts.
Software contracts, as defined by Meyer, are simply an enhanced (and somewhat, object-oriented) version of the pre and post conditions used in algorithm analysis since Donald Knuth.
The promise is exceptional: provide bug-free components, able even to avoid bugs like the one that affected the Arianne 5 crash. Not only: it should allow for an always up-to-date documentation, because it would be an effective part of the object interface.

Unfortunately, the book does nothing to sell really this concept. Instead, it makes it quite clear that you have to use the Eiffel language to unleash all the power of software contracts.
Well, sort of.
Dynamic languages allows for several extensions to the language, without the need of preprocessors. It is thus possible to add some DBC, like the Pre-Post-Conditions in the Python decorator Library.
Unfortunately, although not as cumbersome as using a preprocessor, these methods are not exactly a complete DBC implementation; moreover, unit tests offer a similar confidence in the software, while much easier to introduce as a development practice.

That said, I would really recommend the book to anyone willing to improve his design skills.
DBC require a completely different design perspective, focusing on a programming style that, by separating queries by operation stress particularly on avoiding side effects, and on a programming style that can easily allow for parallel processing. If it may seems too abstract, the authors show it in a very concrete way, with a complete set of principles and guidelines, and full examples on how to apply them.

The book is also a very good introduction to a terminology (software contracts) that proves very effective expressing concepts and designs – or to find flaws during a review.
Actually, I have found this second aspect even more compelling than the previous. Once the basic idea of software contract is well defined, analysis of interfaces, API and designs is definitely easier; and although these are concepts quite a lot in use nowadays, I still think that an operative definition (even by means of techniques and languages not used in the project) helps quite a lot in their usage and communication.

Design by Contract, by Example
ISBN: 0201634600
ISBN-13: 9780201634600

Passwords should always be stored as hashes (actually, they should be salted and hashed, but that’s another story); it should be common knowledge, now.
Yet, several websites, even quite big ones, store passwords as plaintext.

For this reason, always ask for a password reminder. If they will send you your old password (no matter how many checks they will put), then you know they are storing it as plaintext. And that anyone is virtually able to eventually get it, and use it.

Everybody knows what unit testing is.
Well, maybe not. Actually, in most environments, unit tests are a kind of empty “enterprise” word.

Unit test, in their “purest” idea, should test small unit of code (a function, an object, a file, a package) in perfect isolation; usually, this requires the use of stubs or other kind of fake objects.
Of course, we have to be pragmatic; sometimes, a unit is somewhat bigger than it should, and often it unit will embed a complex system, like a database.

In any case, the main advantage of a unit test is that it exactly pinpoints the source of the error, without the need of complex analysis or debugging sessions. This advantage obviously decreases with the complexity of the system.

That said, unit testing is only one of the possible test practices. With some simplification, we may classify them in the following way:

• Unit tests
  Focus is on a development unit and its interface.
• Integration tests
  Focus is on integration between components (like a class and a database) and their integration.
• Functional tests
  Focus is on complete functionalities of the program, from a user-perspective.

All of these tests have one trait in common: they are developer tests, or tests written by software developers.
We need functional tests, in order to check a feature in its “real” environment.
We need integration tests, in order to work out all possible issues deriving from integrations.
We need unit tests, because they are able to pinpoint exactly an error in a very small unit of code.

All those tests help in raising software quality, and for this reason we should go for all of them, whenever possible.
Of course, this approach may be quite expensive; therefore, we should always have a clear idea of testing scope, in particular what we should test, and when we should run a test.
JUnit website makes it quite clear: we should test what could reasonably break, and we should run tests every time code is changed.
How this translates into actual projects, depend on several factors; my own suggestions is to take it as literal as possible, because it will greatly enhance the developer confidence in the final code, and finally lower the total effort for software development and maintenance.

Some further reading:

• Unit test definition on C&C wiki
• Michael Feathers on unit test execution speed
• List of anti-patterns for tests and test suites
• GeoSoft.no unit testing guidelines
• Google testing blog

First googling: set REPLICATOR_SYNC_UNREAD=-1 in notes.ini. From documentation, this may have a performance drawback; in any case, it is not working.
Second googling: starting from Notes 6.03, unread marks are actually synchronized between replicas, and the setting above is no longer used (a-ah!). There is some trick to force replication of unread marks when doing transition from previous releases, but unfortunately this does not solve my problem.

A better read of documentation, and finally I understand the issue: when migrating to the new PC, I simply copied all of my data. Thus, I do not have two PC with different replicas, but two copies of the same replica.
In order to avoid un-necessary traffic, a replica never synchronizes with its own changes on unread marks; as the two replicas were actually the same, they were not exchanging that kind of information.

Solution was then quite easy: simply create a new replica on one of the two PC and then discard the old one.
Of course, I could have avoided all of this if I would have created a new replica from the beginning, instead of lazily copying the same.

More information on replica and unread marks:

• IBM – The Architecture of Unread Marks in Lotus Notes
• IBM – The enhanced unread functionality introduced in Notes/Domino 6.0.3 and 6.5

In my settings, I set hibernation after 45 minutes of stand-by and on push of the sleep button (in case of my Dell, Fn-Esc).
Unfortunately, it was not working: when requesting to Hibernate, it was using a bit the hard drive, then stopped with the quite cryptic message “Insufficient System Resources Exist to Complete the API”. Furthermore, after this error the Hibernate command and options disappeared from the system until next restart.

After some googling, I discovered that it is a common error, already fixed and documented in the Knowledge Base (article).
It happens on systems with more than 1 GB of RAM, and the latest version of Windows XP (SP2 and Media Center 2005): under certain configurations, the system is not able to obtain enough memory for the Hibernation process, thus aborting it.
Note: a colleague hinted the patch is also available within service pack 3.

The fix is easy: update to SP3 or install the patch (from the article above or directly from here), restart the PC, and live happy – and hibernated.

[updated on May 28, 2008]

Here is a very rough instruction for its usage (more details):

1. Add the directory containing the module pydevd to the PYTHONPATH environment variable. Note: actual path may change, reflecting the Pydev version installed.
     set PYTHONPATH=%PYTHONPATH%;C:\eclipse\plugins\org.python.pydev.debug_1.3.14\pysrc
2. From Eclipse, start Python debugging server
3. Add the following instructions to the python script
     import pydevd
     pydevd.settrace()
4. Execute the script; when hitting the lines above, it will break and connect to Eclipse environment. From there on, you can set breakpoints, inspect variables, and use all standard debugging techniques.

In my opinion, these steps are a bit cumbersome: especially changing the PYTHONPATH sounds a bit as a hack, and actually may be even impossible, for instance, when using python within a service.
However, regarding the last point there is a solution: because while importing modules Python uses the sys.path list, which is in turn initialized from PYTHONPATH environment variable.
Knowing that, here is a slightly longer suggestion that allows for including pydevd without any modification of the environment:

def pydevBrk():
  pydevdPath = r“C:\eclipse\plugins\org.python.pydev.debug_1.3.14\pysrc”
  import sys
  if not pydevdPath in sys.path:
    sys.path.append(pydevdPath)
  try:
    import pydevd
    pydevd.settrace()
  except ImportError:
    pass # Most probably, pydev is not installed on the system

A small advantage of this variant, it fail gracefully even in case pydev is not installed on the system – in other words, if for some reason debug code makes its road to production, it will not crash the entire application.
Of course, in this case the initial breakpoint will stop inside the function, and not where it is being called.

A final note about pydevd.settrace(): there are several parameters, which may be very useful, as the described in the docstring:

@param host: the user may specify another host, if the debug
    server is not in the same machine
@param stdoutToServer: when this is true, the stdout is passed
    to the debug server
@param stderrToServer: when this is true, the stderr is passed
    to the debug server so that they are printed in its
    console and not in this process console.
@param suspend: whether a breakpoint should be emulated as
    soon as this function is called.

Note: Remote debugging requires some further setup, in order to translate paths from the application to the server; this requires editing the module pydevd_file_utils (in the same folder as pydevd), and setting the translation table PATHS_FROM_CLIENT_TO_SERVER; although this is quite an easy and straightforward step, it has to be repeated for each new version of the plugin.

Use real-size test data for development

Development is usually performed on ad-hoc databases, with few items created just for basic testing.
While this may be a useful tool for correctness tests, you will hardly ever find performance issues. Ok, you may anticipate them and add index to database at will, but there is no general recipe for performance; and without a good data set, such issues sooner or later will appear.
Only testing with real-size data allows for performance findings.
Note that real data means also real use cases, with target performance: good performance is only useful as long as customer appreciate it, otherwise is a worthless effort.
Therefore, once equipped with a real data sets and scenarios, start profiling. Measure the performance, and make a plan. Where is the bottleneck? Is it possible to optimize that part? What is the current gap in comparison to the target?

Assess complexity of every class and methods

While the former approach is correct, I think we should also make a small pro actively step back, at our programming courses, when we have been taught about algorithm correctness and complexity, and how to assess it. Yep, this is a kind of lost knowledge: with the notable exception of STL, it is quite uncommon to see such analysis in the code or in its documentation.
This should not be an obstacle though, and a very useful tool for tracking complexity is the so called “Big-O notation“.
Wikipedia has quite a good explanation, but roughly speaking, this notation gives an idea of algorithm complexity as function of the size of the input data. For instance, a linear algorithm will require about double of the time to complete for a double set of data, while a quadratic one will require four times, and a constant one will return a result in a constant time, no matter the size or the kind of the input data.
It is an invaluable tool for assessing algorithm complexity; unfortunately it is not always so easy to use: a good analysis may involve calculation of series, and special thinking on statistical data; but in many cases, even a wrong, pessimistic, calculation is already a good starting point.
Of the various complexity classes (again, take a look at Wikipedia article for a complete listing) the most important are (from “faster” to “slower”):

• Constant
• Logarithmic
• Linear
• Quadratic
• Polynomial
• Exponential

Generally speaking, you should try to avoid everything more complex than linear. Of course, this is not always possible, but then you should beware large sets of data.

Be humble

When your application is slow, don’t blame the operating system, the language or third party libraries.
Don’t even say “it is an heavy calculation, hence it must be slow”.
You can say whatever you want, but your application will still be slow.
Instead, assess performance, ask for a target, and start profiling. Analyze and then try to remove unnecessary complexities.
Be humble, and fix your errors.

For instance, garbage collection.

Mainly, Python memory management is implemented through reference counting: as soon as the number of references of an object reaches zero, it is deleted. What “delete” means, then, depends on the actual implementation of the python VM.
CPython, for instance, uses delete the object: this means that the C++ destructor (or, in case of Python objects, the __del__ method) is called, and the memory may be released (or may return to a common pool handled directly by the python memory manager, in case of “simple” objects like integers); in other words, CPython uses deterministic finalizers.
However, this behavior is not guaranteed on other implementations, like JPython or IronPython, because of the different underlying memory model.

Reference counting has several advantages because it is easy to implement, fast, and predictable; on the other hand, it is not able to handle some cases, in particular circular references.
Circular references happens when items in a container maintain a reference to the container itself; in this particular situation, when the container goes out of scope its reference count will still remain higher than zero, and hence will cause a memory leak.
For this reason, since Python 2.0, there is a new module, called gc, that perform some garbage collecting.
Its main and sole purpose is to handle those particular situations, and does not change nothing on standard memory management.

Back to our initial problem, this is quite a big difference with, for instance, C#, where calling GC.Collect() will effectively deallocate all pending objects. In python, gc.collect() will simply run a “check” for circular references, and deallocate if necessary all pending object in such state.

This is the reason why, in case of problems with memory deallocation, calling gc.collect() in python is in most cases almost useless.
In python, such problems are symptoms of flaws in the design, and blaming the language because it does not behave like other certainly do not help fixing them.

1. Trying to guess future business cases is job for Program Managers, not developers.
2. The simplest design that fits the solution is usually the best you can conceive.
3. Refactoring is a developer best friend; in case future requirements will require an extension of the design, simply do it.
4. Software can (and should) change over time.

Nevertheless, this is something difficult to achieve.
How many times we put a singleton, simply because we liked the pattern?
How many times we tried to anticipate the needs of a library, spending quite big amounts of effort in developing a flexible and extensible architecture that no one but us would use?
We keep creating something flexible and powerful, which in most cases is simply a waste of time; moreover it leads to over-complicated designs, with lower performance and difficult to maintain.
And very often all that flexibility is even not used at all!

I think, like many other Agile methodologies it is a way of thinking.
Because we all know that interfaces are sacred cows of design, we want them to be right at th first iteration, and not to change them.
Because we are going to design a piece of software that will stay there in the core of the system, we want to prefigure out as many possible use cases as possible.
Because we love what we do, we want to think that it will be so clever that no one will attempt to (radically) change it.

And we forget that interfaces may be changed (or added), that refactoring is our best friend and that in general software is so easy to change.

Here are a couple of tricks to reduce their size:

1. Use a database!
   Very often, excel is used s a database. Please, don’t do that; if you really need a database, even Access is a better choice than a bunch of Excel sheets – for a good amount of reasons.
2. Using pivot tables, un-checking the option “Save data with table layout” (right-click on the entry, then select “Table options”) will definitely help a lot. Of course, use this option with care when dealing with external data.
3. Split content in several files.
   Sometimes, an excel file may grow of its original scope, and incorporating useless data. Just a short analysis may help into splitting different data and views into different sheets.
4. Remove empty cells.
   As a result of several editing on a file, or simply because of a mistakenly change in a cell at row 48983, you may end up with a sheet with thousands of empty (non-Null) cells.
   To remove them, simply select all rows from the last used one on, and then delete them (the same apply to columns), then save the file.
5. Compact macros.
   Macros may take up quite a lot of space, and become very fragmented.
   In order to recover that space, export all macros, clean up all code, save and finally paste back the exported modules; this operation will very often free quite a nice amount of space.

Note: all of these suggestions apply to Excel up to version 2003; I have never tested them with the new XML format introduced with Office 2007.

[updated on March 28, 2008]

Their syntax is quite simple, and coherent with other python structures:
lambda x: expression(x)

that is equivalent to
def anon(x):
return expression(x)

Note that, although the name, goal, python lambda expressions are not equivalent to their synonyms in functional languages because of three main limitations, and thus cannot be used for “serious” functional programming:

• Lambda functions may only contain expressions (no statements, hence no if/then/else constructs, for example).
• Lambda functions are made of exactly one expression.
• Lambda functions do not have any local storage.

That said, there are several tricks that will somewhat allow to overcome the above limitations; as all tricks, they should be used with extreme care.

So, when should we use lambda functions? Well, better never than often, and most of the times using lambda expressions is not a so good idea.
Because, although geek and cool, lambda expressions may seriously hinder code readability.

As a general rule, better avoid use of lambda expressions. There may be cases where having an anonymous function may be handy (like in a property getter/setter), but readability is much more important than a cool and compact code style.