Eclipse HTML editor

Most of the time I really love Eclipse; however, sometimes, I really hate it.
Currently, I am still using Eclipse 3.3, and as I am doing some “web” development with Django,  I thought I could use WST (Web Standard Tools). However, I could not use it: downloaded, installed, enabled – no chance to see any difference in the system. I am quite sure I am doing something wrong, but, and this is what I really hate in Eclipse, I was not able to find any simple “how to” documentation that could help me in solving the issue.

However, the plugin ecosystem is healthy, and I was then able to find a good replacement in Amateras HTML Editor Plugin. I quite like it, and even if I have to edit the markup code, I can still see the preview, so it is not so bad after all. And the other tools (like palette, and especially the css editor) are well designed, and helps a lot. Yes, that’s not Dreamweaver, but for the moment I can live with that.

Another option, finally, is Aptana Studio. I am using it right now, and it’s better than Amateras. However, it is more invasive, and it has a slight “adware” attitude, because it has full of links for the “extended” version of the plugin, that requires some form of subscription. Nevertheless, it allows for a far better preview of html files, and if you do not care for those links, or (like me) avoid using the Aptana perspective, then it is quite OK: it still forces you to work on source level, but it has a good amount of helpers and the preview works enough to be useful.

import this

The Zen of Python, by Tim Peters

Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren’t special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one– and preferably only one –obvious way to do it.
Although that way may not be obvious at first unless you’re Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it’s a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea — let’s do more of those!

Fun to speech

import win32com.client
reader = win32com.client.Dispatch("sapi.spvoice")
reader.Speak ("hello, world")

As a matter of facts, this trick is pretty old – at least, as old as Windows XP.
But it is the kind of discover that amuses me like a little boy :-)

Reading list: Design by contract, by Example

Design by contract, by example is a kind of nice book.
I mean, for me, it failed completely to the goal of pushing towards design by contract, and still was very useful: by stressing on immutable classes and some sound design principles guidelines, and by clarifying a language – functions and objects contracts – that is becoming more and more common because of its expressiveness of responsibilities in interfaces.

I bought it because I was interested in software contracts.
Software contracts, as defined by Meyer, are simply an enhanced (and somewhat, object-oriented) version of the pre and post conditions used in algorithm analysis since Donald Knuth.
The promise is exceptional: provide bug-free components, able even to avoid bugs like the one that affected the Arianne 5 crash. Not only: it should allow for an always up-to-date documentation, because it would be an effective part of the object interface.

Unfortunately, the book does nothing to sell really this concept. Instead, it makes it quite clear that you have to use the Eiffel language to unleash all the power of software contracts.
Well, sort of.
Dynamic languages allows for several extensions to the language, without the need of preprocessors. It is thus possible to add some DBC, like the Pre-Post-Conditions in the Python decorator Library.
Unfortunately, although not as cumbersome as using a preprocessor, these methods are not exactly a complete DBC implementation; moreover, unit tests offer a similar confidence in the software, while much easier to introduce as a development practice.

That said, I would really recommend the book to anyone willing to improve his design skills.
DBC require a completely different design perspective, focusing on a programming style that, by separating queries by operation stress particularly on avoiding side effects, and on a programming style that can easily allow for parallel processing. If it may seems too abstract, the authors show it in a very concrete way, with a complete set of principles and guidelines, and full examples on how to apply them.

The book is also a very good introduction to a terminology (software contracts) that proves very effective expressing concepts and designs – or to find flaws during a review.
Actually, I have found this second aspect even more compelling than the previous. Once the basic idea of software contract is well defined, analysis of interfaces, API and designs is definitely easier; and although these are concepts quite a lot in use nowadays, I still think that an operative definition (even by means of techniques and languages not used in the project) helps quite a lot in their usage and communication.

Design by Contract, by Example
ISBN: 0201634600
ISBN-13: 9780201634600

Never trust who sends you your password

Because in order to send you your password, they need to know it.

Passwords should always be stored as hashes (actually, they should be salted and hashed, but that’s another story); it should be common knowledge, now.
Yet, several websites, even quite big ones, store passwords as plaintext.

For this reason, always ask for a password reminder. If they will send you your old password (no matter how many checks they will put), then you know they are storing it as plaintext. And that anyone is virtually able to eventually get it, and use it.

Developer tests

Everybody knows what unit testing is.
Well, maybe not. Actually, in most environments, unit tests are a kind of empty “enterprise” word.

Unit test, in their “purest” idea, should test small unit of code (a function, an object, a file, a package) in perfect isolation; usually, this requires the use of stubs or other kind of fake objects.
Of course, we have to be pragmatic; sometimes, a unit is somewhat bigger than it should, and often it unit will embed a complex system, like a database.

In any case, the main advantage of a unit test is that it exactly pinpoints the source of the error, without the need of complex analysis or debugging sessions. This advantage obviously decreases with the complexity of the system.

That said, unit testing is only one of the possible test practices. With some simplification, we may classify them in the following way:

• Unit tests
  Focus is on a development unit and its interface.
• Integration tests
  Focus is on integration between components (like a class and a database) and their integration.
• Functional tests
  Focus is on complete functionalities of the program, from a user-perspective.

All of these tests have one trait in common: they are developer tests, or tests written by software developers.
We need functional tests, in order to check a feature in its “real” environment.
We need integration tests, in order to work out all possible issues deriving from integrations.
We need unit tests, because they are able to pinpoint exactly an error in a very small unit of code.

All those tests help in raising software quality, and for this reason we should go for all of them, whenever possible.
Of course, this approach may be quite expensive; therefore, we should always have a clear idea of testing scope, in particular what we should test, and when we should run a test.
JUnit website makes it quite clear: we should test what could reasonably break, and we should run tests every time code is changed.
How this translates into actual projects, depend on several factors; my own suggestions is to take it as literal as possible, because it will greatly enhance the developer confidence in the final code, and finally lower the total effort for software development and maintenance.

Some further reading:

• Unit test definition on C&C wiki
• Michael Feathers on unit test execution speed
• List of anti-patterns for tests and test suites
• GeoSoft.no unit testing guidelines
• Google testing blog

Lotus Notes: replicating unread marks across different clients

As I have started using a notebook in addition to my PC, I have also started having issues with replicas of my email.
In particular, my main issue is unread marks replication: it is simply not working across the two workstations, and I have often to re-read mails over again, just to figure out if I have already replied them.

First googling: set REPLICATOR_SYNC_UNREAD=-1 in notes.ini. From documentation, this may have a performance drawback; in any case, it is not working.
Second googling: starting from Notes 6.03, unread marks are actually synchronized between replicas, and the setting above is no longer used (a-ah!). There is some trick to force replication of unread marks when doing transition from previous releases, but unfortunately this does not solve my problem.

A better read of documentation, and finally I understand the issue: when migrating to the new PC, I simply copied all of my data. Thus, I do not have two PC with different replicas, but two copies of the same replica.
In order to avoid un-necessary traffic, a replica never synchronizes with its own changes on unread marks; as the two replicas were actually the same, they were not exchanging that kind of information.

Solution was then quite easy: simply create a new replica on one of the two PC and then discard the old one.
Of course, I could have avoided all of this if I would have created a new replica from the beginning, instead of lazily copying the same.

More information on replica and unread marks:

• IBM – The Architecture of Unread Marks in Lotus Notes
• IBM – The enhanced unread functionality introduced in Notes/Domino 6.0.3 and 6.5

Hibernate not working on Windows XP

A couple of weeks ago, I received a new notebook (Latitude D830), as per company policy still equipped with Windows XP SP2.
As usual for me, I immediately started playing with power settings, especially regarding Sleep and Hibernation.
I like Hibernation: it allows for actually turning off the computer while still maintaining the current state. Of course, this is no substitute for saving files (what if the system cannot restore?) but it is much easier and faster than each time manually restoring all open windows and files.

In my settings, I set hibernation after 45 minutes of stand-by and on push of the sleep button (in case of my Dell, Fn-Esc).
Unfortunately, it was not working: when requesting to Hibernate, it was using a bit the hard drive, then stopped with the quite cryptic message “Insufficient System Resources Exist to Complete the API”. Furthermore, after this error the Hibernate command and options disappeared from the system until next restart.

After some googling, I discovered that it is a common error, already fixed and documented in the Knowledge Base (article).
It happens on systems with more than 1 GB of RAM, and the latest version of Windows XP (SP2 and Media Center 2005): under certain configurations, the system is not able to obtain enough memory for the Hibernation process, thus aborting it.
Note: a colleague hinted the patch is also available within service pack 3.

The fix is easy: update to SP3 or install the patch (from the article above or directly from here), restart the PC, and live happy – and hibernated.

[updated on May 28, 2008]

Debugging with Pydev Extension

Without any doubt, Pydev Extension is a wonderful environment for Python development.
One of the nicest features is the “remote debugger”, that allows attaching to every python script, even if it was not launched from Eclipse; actually, where I work this was the “killer feat” that convinced management over buying it.

Here is a very rough instruction for its usage (more details):

1. Add the directory containing the module pydevd to the PYTHONPATH environment variable. Note: actual path may change, reflecting the Pydev version installed.
     set PYTHONPATH=%PYTHONPATH%;C:\eclipse\plugins\org.python.pydev.debug_1.3.14\pysrc
2. From Eclipse, start Python debugging server
3. Add the following instructions to the python script
     import pydevd
     pydevd.settrace()
4. Execute the script; when hitting the lines above, it will break and connect to Eclipse environment. From there on, you can set breakpoints, inspect variables, and use all standard debugging techniques.

In my opinion, these steps are a bit cumbersome: especially changing the PYTHONPATH sounds a bit as a hack, and actually may be even impossible, for instance, when using python within a service.
However, regarding the last point there is a solution: because while importing modules Python uses the sys.path list, which is in turn initialized from PYTHONPATH environment variable.
Knowing that, here is a slightly longer suggestion that allows for including pydevd without any modification of the environment:

def pydevBrk():
  pydevdPath = r“C:\eclipse\plugins\org.python.pydev.debug_1.3.14\pysrc”
  import sys
  if not pydevdPath in sys.path:
    sys.path.append(pydevdPath)
  try:
    import pydevd
    pydevd.settrace()
  except ImportError:
    pass # Most probably, pydev is not installed on the system

A small advantage of this variant, it fail gracefully even in case pydev is not installed on the system – in other words, if for some reason debug code makes its road to production, it will not crash the entire application.
Of course, in this case the initial breakpoint will stop inside the function, and not where it is being called.

A final note about pydevd.settrace(): there are several parameters, which may be very useful, as the described in the docstring:

@param host: the user may specify another host, if the debug
    server is not in the same machine
@param stdoutToServer: when this is true, the stdout is passed
    to the debug server
@param stderrToServer: when this is true, the stderr is passed
    to the debug server so that they are printed in its
    console and not in this process console.
@param suspend: whether a breakpoint should be emulated as
    soon as this function is called.

Note: Remote debugging requires some further setup, in order to translate paths from the application to the server; this requires editing the module pydevd_file_utils (in the same folder as pydevd), and setting the translation table PATHS_FROM_CLIENT_TO_SERVER; although this is quite an easy and straightforward step, it has to be repeated for each new version of the plugin.

Scaling performance

It is quite common to see software that does not scales well.
Software that works well on a developer machine, and suddenly it start crawling when using real data.
I have seen it in my teams, and usually it escalates to a critical emergency: all other development stops, until a solution is found, if any.

Use real-size test data for development

Development is usually performed on ad-hoc databases, with few items created just for basic testing.
While this may be a useful tool for correctness tests, you will hardly ever find performance issues. Ok, you may anticipate them and add index to database at will, but there is no general recipe for performance; and without a good data set, such issues sooner or later will appear.
Only testing with real-size data allows for performance findings.
Note that real data means also real use cases, with target performance: good performance is only useful as long as customer appreciate it, otherwise is a worthless effort.
Therefore, once equipped with a real data sets and scenarios, start profiling. Measure the performance, and make a plan. Where is the bottleneck? Is it possible to optimize that part? What is the current gap in comparison to the target?

Assess complexity of every class and methods

While the former approach is correct, I think we should also make a small pro actively step back, at our programming courses, when we have been taught about algorithm correctness and complexity, and how to assess it. Yep, this is a kind of lost knowledge: with the notable exception of STL, it is quite uncommon to see such analysis in the code or in its documentation.
This should not be an obstacle though, and a very useful tool for tracking complexity is the so called “Big-O notation“.
Wikipedia has quite a good explanation, but roughly speaking, this notation gives an idea of algorithm complexity as function of the size of the input data. For instance, a linear algorithm will require about double of the time to complete for a double set of data, while a quadratic one will require four times, and a constant one will return a result in a constant time, no matter the size or the kind of the input data.
It is an invaluable tool for assessing algorithm complexity; unfortunately it is not always so easy to use: a good analysis may involve calculation of series, and special thinking on statistical data; but in many cases, even a wrong, pessimistic, calculation is already a good starting point.
Of the various complexity classes (again, take a look at Wikipedia article for a complete listing) the most important are (from “faster” to “slower”):

• Constant
• Logarithmic
• Linear
• Quadratic
• Polynomial
• Exponential

Generally speaking, you should try to avoid everything more complex than linear. Of course, this is not always possible, but then you should beware large sets of data.

Be humble

When your application is slow, don’t blame the operating system, the language or third party libraries.
Don’t even say “it is an heavy calculation, hence it must be slow”.
You can say whatever you want, but your application will still be slow.
Instead, assess performance, ask for a target, and start profiling. Analyze and then try to remove unnecessary complexities.
Be humble, and fix your errors.