Dariusz Dwornikowski

Windows 2008 on KVM no disk solved (proxmox)

2010-10-15T18:12:00.000+02:00

If you have problem finding it - windows 2008 does not play well with SCSI virtual disk in KVM. On the other hand, for VIRTIO you have to point Windows to additional driver iso with VIRTIO drivers. You can find them here.

Thanks to Proxmox VE support forum for help.

Dymo LabelWriter 400 on ubuntu

2010-09-30T19:32:00.000+02:00

Dymo LabelWriter 400

Today I bought Dymo LabelWriter 400 printer for labels. Surprisingly it works like a charm on my ubuntu machine. It gets recognised as normal printer, all you have to do is to download cups driver from Dymo page and compile it. (do not forget to apt-get install libcups2-dev and libcupsimage2-dev).

For a simple scenarios (nice word for lame label printing) I use http://www.glabels.org/, buy my ultimate aim is to create Django app to store information about various devices we own in our research group. The problem on out university is that we cannot easily get rid of old assets, as the procedures take time and are painful (as in plenty-papers-forms-filling painful). That is why one can find SCO server from the 80s, many types of handhelds (like 1337 Sharp Zaurus - you all remember the fuss) and even super size IBM supercomputer with computing power equal to Pentium 100 (literally).

Ok back to Dymo gadget. So my aim is to create and print labels from Django (Python) dynamically. And here I have a problem. I can see the paper size I use is "Large Address" 89x36mm size. When I try to print an image of that size (Gimp created) it results in taking only circa 2/3 of the label. I am still trying to figure out how it can be. Maybe image gets transformed into postscript and the size gets smaller ? Also surprisingly cups fit-to-page option does not work.

Anyway. I will try to do it and try to post the result.

Custom registration form with django-registration

2010-09-28T16:40:00.000+02:00

Following StackOverflow question I manage to customise registration form used with djang-registration application.

The SOF answer not explanatory, so here it what you have to do:

1. Copy imports and register method from django-registration default backend (it is in __init__.py) and paste it in step 3.
2. Create your form, example below:

from registration.forms import RegistrationForm

...
class UserRegistrationForm(RegistrationForm):
        first_name = forms.CharField()
        last_name = forms.CharField()

3. Create a file in your APP HOME, in which you implement your own backend. I used regbackend.py, as in the example. Paste imports and register method from step 1, see code below:

from django.conf import settings
from django.contrib.sites.models import RequestSite
from django.contrib.sites.models import Site
from django.contrib.auth.models import User
from registration import signals
from PROJECT.APP.forms import UserRegistrationForm
from PROJECT.APP.models import *
from registration.models import RegistrationProfile
class MyBackend(object):

   def register(self, request, **kwargs):
        username, email, password = kwargs['username'],kwargs['email'], kwargs['password1']
        if Site._meta.installed:
            site = Site.objects.get_current()
        else:
            site = RequestSite(request)
        new_user = RegistrationProfile.objects.create_inactive_user(username, email,
                                                                    password, site)
        signals.user_registered.send(sender=self.__class__,
                                     user=new_user,
                                     request=request)
        user = User.objects.get(username=username)
        user.first_name=kwargs['first_name']
        user.last_name=kwargs['last_name']
        address_user = Address()
        address_user.save()
        contact_user = Contact(address=address_user,email=user.email)
        contact_user.save()

        address_company = Address()
        address_company.save()
        contact_company = Contact(address=address_company)
        contact_company.save()
        company = Company(contact=contact_company, admin=user)
        company.save()
        user_profile = UserProfile(user=user,contact=contact_user,position='',company=company  )
        user_profile.save()
        user.save()

        return new_user

4. Change your urls.py

import PROJECT.regbackend
from PROJECT.APP.forms import UserRegistrationForm
from registration.views import register
urlpatterns = patterns('',
...
url(r'^register/$', register, { 'backend': 'PROJECT.regbackend.MyBackend' }, name='registration_register'),
...
)

Here you go. Now you can add custom fields to the form. Also you can use form validators from django-registration. They are placed in forms.py in the sources.

Custom js and css in Django forms

2010-09-22T09:17:00.000+02:00

Just to remind myself. If you want to put custom js or css when your form displays, you have to write your own widget. This is obvious. However docs do not say about one important thing.

You have to include {{forms.media}} tag in your template's head.

Django and i18n

2010-09-21T00:21:00.000+02:00

Just a note post to remember how to set up the i18n in Django for the templates.
1. apt-get install gettext
2. Add {% load i18n %} to the top of the templates.
3. Add 'django.middleware.locale.LocaleMiddleware' to MIDDLEWARE_CLASSES
4. Add 'django.core.context_processors.i18n' to TEMPLATE_CONTEXT_PROCESSORS
5. Insert {% trans "VARIABLE_MSG" %} into your templates, where you need the translations.
6. mkdir locale
7. Run python manage.py makemessages -l LANG_CODE -a to generate po files
8. Run python manage.py compilemessages to compile po files into mo files.

That is it.

Simple animations with Beamer overlays

2010-09-17T15:39:00.001+02:00

Sometimes you want to create a simple “animation” in beamer, for example to show step by step transitions in labelled transition system.

In PowerPoint style tool, you would just use the internal drawing capabilities. In Beamer it is not so easy. Fortunately Beamer has overlays, so you can just create many figures, each one representing the next frame in your animation. Without overlays, you would create Beamer frame for each of these picture frame. However this method is not code efficient and you just end with thousands of LaTeX lines. With overlays you can create one beamer frame and control the order in which components are shown.
Have a look at the example. Suppose, I want to show an animation of transitions in a labeled transition system. I created 4 pictures in Dia: step1.pdf, step2.pdf, step3.pdf and step4.pdf. I also want to show appropriate text on each animation frame. So what I have to do is to show step1.pdf only on frame 1, step2.pdf only on frame 2, etc. See the code.

begin{frame}{Steps example}
  \begin{figure}[H]
    \begin{center}
      % \only shows component only on frame X of the animation
      \only<1>{ \includegraphics[width=0.6\textwidth]{step1.pdf}}
      \only<2>{ \includegraphics[width=0.6\textwidth]{step2.pdf}}
      \only<3>{ \includegraphics[width=0.6\textwidth]{step3.pdf}}
      \only<4>{ \includegraphics[width=0.6\textwidth]{step4.pdf}}
    \end{center}
  \end{figure}
 
  \begin{block}{Labels for animation frames}
    \begin{itemize}
      % <1-> will show the text all frames from 1-X, <2-> on frames 2-X and so on
      \item<1-> step 1 is the first 
      \item<2-> then goes step 2
      \item<3-> and step 3
      \item<4-> finally the step 4
    \end{itemize}
  \end{block}
\end{frame}

Custom IMAP in Vim LaTeXSuite

2010-09-17T10:07:00.000+02:00

Vim LaTeXSuite uses IMAPS to ease LaTeX documents creation. For example you can write EFI in insert mode and you automatically get:

\begin{figure}[<+htpb+>]
  \begin{center}
    \includegraphics{<+file+>}
  \end{center}
  \caption{<+caption text+>}
  \label{fig:<+label+>}
\end{figure}<++>

You can navigate through the placeholders (<+somthing+>) with CTRL+J to fill them in. To create custom IMAP, you place it in ~/vim/ftplugin/tex/vim. Example for Beamer frame:

:call IMAP('BFR',"\\begin{frame}\\\frametitle{<++>}\<++>\\\end{frame}", 'tex')

The first argument "BFR" is a name for the IMAP, the second one is a body to place when vim sees BFR typed in. 'tex' word attaches the IMAP only to tex type files.

Hash serialisation in Python

2010-09-17T10:02:00.000+02:00

I needed some serialisation for hashes in Python for my small UNIX userland project. I needed to store name:path pairs in a convenient way. First I thought of Berkeley DB, but saw in the PyDocs that it will be deprecated in Python 3.0. I asked question on the Stackoverflow and got a nice answer. Here is the link to the question.

So SOF users showed me shelve, a python serialisation module, which in fact uses BDB under the hood. The example below is directly taken from the SOF post:

import shelve
d = shelve.open('filename')
d['name'] = 'path'
d.close()

Text above arrows in LaTeX

2010-09-17T09:57:00.000+02:00

Sometimes you need to place text above arrows. Let's say you need to write transitions for a labelled transition system. In fact in LaTeX this is not so obvious, especially if you want to have text above and under the arrow. I know of two easy ways to do it.

First method:

$$ E \xrightarrow{\alpha} F $$

... and the second one.

$$ \mathop{\longrightarrow}^_{\alpha} $$

Proxmox clone virtual machines (with iSCSI scenario)

2010-04-20T00:25:00.000+02:00

Proxmox VE is a great web front/system for underlaying KVM/qemu virtualisation environment. One of its greatest features is that it can work in a cluster mode. Add iSCSI storage to that and You end up with a nice cloud like solution for running your machines. One of flaws of Proxmox is that it does not have ability to clone virtual hard disks, it is especially hard when you use lvm groups for your images. Yesterday I found an easy and working solution for that problem. As always I was driven by a need - a request for 8 identical SUSE systems.

My setup is as follow:
- 4 Proxmox physical nodes connected into a cluster,
- 1 BIG F** LUN on iSCSI (4TB),
- every virtual disk is a LVM volume in this LUN (ofc Ive got an LVM Group too),
- hard disks in raw format.

The solution is easy as hell.

1. You create a source/template virtual machine and install the system.
2. You create virtual machines with exactly the same hd size as the source machine.
3. You issue dd if=/dev/lvmgroup/vm-{$VMID_SOURCE}-disk-1 of=/dev/lvmgroup/vm-{$VMID_NEW}-disk-1
4. You jump start new vms and configure unique IP address for them.
5. Done :)

PS: Ofc. {$VMID_SOURCE} is the ID for the source virtual machine, and {$VMID_NEW} is the id ofor the clone.

Custom kernel on Proxmox host

2009-11-23T15:23:00.001+01:00

Proxmox is a great platform for KVM/kqemu and OpenVZ virtualization. It is generally a debian based system with web panel, that controls the creation and management of your virtual machines. It bases though on a little bit oldish kernel 2.6.24, due to support for openVZ. Normally it is not a problem, but in my server's case I needed kernel 2.6.31 to support newer hardware.

In order to make proxmox work on vanilla kernel, one needs only to be sure, that KVM support is compiled in. So the first step is to compile kernel and reboot. The machines won't start, because of the lack of fairsched, saying: unable to create fairsched node. In order to get rid of the message and make your machines work on vanilla kernel, you need to edit proxmox module called QemuServer.pm in /usr/share/perl5/PVE directory.
Find the:

if (!$use_virtio) {
   push @$cmd, '-id', $vmid;
   my $cpuunits = defined ($conf->{cpuunits}) ?
   $conf->{cpuunits} : $defaults->{cpuunits};
   push @$cmd, '-cpuunits', $cpuunits if $cpuunits;
   # fixme: cpulimit is currently ignored
   #push @$cmd, '-cpulimit', $conf->{cpulimit} if $conf->{cpulimit};
}

and comment out everything in the if block. Now the machines can start.

Time Series and the whisper of the universe

2009-07-22T10:52:00.004+02:00

For some time now, I am investigating prediction in time series, in the context of proactive monitoring of distributed systems. So in general lots of math, statistics and R language. An additive model of time series defines that time series is a:

Ts = T + S + Ls + E,

where:
T - is some trend,
S - seasonal nonrandom long term cyclic influence,
Ls - seasonal nonrandom short term cyclic influences,
E - random variable, deviations from ideal non stochastic model.

I like to call E the whisper of the universe, as this is all the garbage left, after decomposition. For example in time series, that reflect the stock price, these can be influence of the sexual scandals of board members :) Whereas in computer systems, electrical current fluctuations, weather impact -> sun explosions :)

The words are not enough, so please look at the plot generated in R by doing stl() function, that decomposes the time series. The remainder is the thing I am talking about.

OpenSSH <= 0day exploit

2009-07-20T14:17:00.002+02:00

Just to inform, http://lists.grok.org.uk/pipermail/full-disclosure/2009-July/069752.html. Anti-sec group posted this threat thread. If it is fake I lost 5min for firewall configuration, if not Ill loose week for restore procedures.

Java Coding Convention

2009-07-07T21:09:00.002+02:00

I was thinking how to force students in the project to write elegant code. I gave them link to SUN's java code convention, but this won't "force" them. So I did a little research on the topic and found a perfect solution for Eclipse. It is called Eclipse-CS and it is a plugin for Eclipse which makes code checks with the help of Checkstyle - open source code style checker. Now I can force beautiful code. Will it be also usable ? I hope so.

Maven, Jersey, Tomcat = RESTful web service

2009-06-12T20:49:00.008+02:00

I'm investigating the topic of monitoring and management of RESTful web service right now, so I decided to actually build one. Following my friend's advice I chose Jersey for my REST library. After close look development with it is fun and easy. I found a Japod's post about how to run jersey with maven in tomcat (in fact you can substitute tomcat for jetty too or whatever you like ). The blog post is quite old, some things are different now, so here is my pom.xml for latest jersey.


4.0.0
my.example.com
SimpleRESTApp
war
1.0-SNAPSHOT
SimpleRESTApp Maven Webapp
http://maven.apache.org


maven2-repository.dev.java.net
Java.net Repository for Maven
http://download.java.net/maven/2/
default




asm
asm
3.1


junit
junit
3.8.1
test



com.sun.jersey
jersey-bundle
1.1.0-ea



SimpleRESTApp


org.codehaus.mojo
tomcat-maven-plugin


maven-compiler-plugin

1.5
1.5

Here I had to add explicitely:


asm
asm
3.1

Then I edited the web.xml generated by maven, as shown on the Japod's blog.


Archetype Created Web Application

Jersey Web Application
com.sun.jersey.spi.container.servlet.ServletContainer

com.sun.jersey.config.property.resourceConfigClasscom.sun.jersey.api.core.PackagesResourceConfig

com.sun.jersey.config.property.packagescom.example.my
1


Jersey Web Application
/*

After that you only have to wrote your example POJO object like that:

package com.example.my;
import javax.ws.rs.GET;
import javax.ws.rs.Produces;
import javax.ws.rs.Path;

@Path("/hello/")
public class SimpleResource {

@GET
@Produces("text/plain")
public String getGreeting() {
return "Hi ! ";
}
}

The only difference in the new version is @Produces, instead of @ProduceMime. Also the import statement is changed. Now type mvn tomcat:run or mvn jetty:run and voila.

Tools for testing web applications

2009-06-03T11:17:00.002+02:00

JMeter:
URL: http://jakarta.apache.org/jmeter/index.html
Language: Java
Features: GUI, jmeter-server, easy scenarios

SLAMD:
URL: http://www.slamd.com/ ( Distributed Load Generation Engine )
Language: Java
Features: WEB GUI, distributed, simple monitoring abilities

Web Driver:
URL: http://code.google.com/p/webdriver/
Language: Java
Features: nice documentation, extensible, emulation of selenium, web browser emulation

Selenium:
URL: http://seleniumhq.org/
Language: XUL?
Features: Firefox addon

Pylot:
URL: http://www.pylot.org/ (Performance & Scalability Testing - Web Services)
Language: Python
Features: Web Gui, SOAP

What is this ESB ?

2009-06-02T21:38:00.002+02:00

Today I gave a seminar lecture on ESB (Enterprise Service Bus) and JBI (Java Business Integrations) on example of Apache ServiceMix. There were many questions concerning the purpose of using ESBs in SOA environments, but the most important was : "What really is ESB usable for ?". I think that the lecture by Mark Richards recorder at NFJS can be the answer and a nice intro to ESB middleware. Watch the presentation here.

Backups with rdiff-backup

2009-04-26T11:36:00.002+02:00

Recently I had to switch my small backup system from bacula to some other solution. The reason was that for relatively small environments, bacula was too problematic. Here is why:

big load on db server ( postgresql )
db was about 19GB, indexing, vacuums were hell
problematic recoveries, bacula keeps it with own format
checking for files was too energy consuming for admins, bconsole, search, etc
the backup jobs lasted too long, keeping the internal backbone occupied
surprisingly bacula was not storage efficient, it took whole backup server space, with only 1month history ( the policy was tuned, I did not store stupid things, only crucial data )
summing up: I had problematic solution, that took 100% of storage space with only 1 month backup history

Before I introduce the solution, one has to understand the infrastructure I am talking about. It is a very small setup in my opinion ( I saw bigger with bacula live). I have got 12 backed up servers and one storage server with 2TB space. The whole rack runs Debians etch and lenny. No other oses. The internal network is 1Gbit with vlans, in way that every server is in separate VLAN, using separate NIC to communicate with storage server.

What I wanted to achieve with new backup system?

Firstly I needed solution that will solve the problems that occurred with existing bacula system. In best scenario I had in mind some differential, fast backup system with clear insight in stored files. The natural solution seemed to be rsync or similar. I did an enquiry on google and found custom crafted solutions how to make differential backups with rsync and hardlinking. I was a little bit afraid to use it on production environment. One day though one of friends on #pplug channel sugested rdiff-backup, an application that did what I needed and more, as it appeared.

What is rdiff-backup ? On its homepage we can read:

rdiff-backup backs up one directory to another, possibly over a network. The target directory ends up a copy of the source directory, but extra reverse diffs are stored in a special subdirectory of that target directory, so you can still recover files lost some time ago. The idea is to combine the best features of a mirror and an incremental backup. rdiff-backup also preserves subdirectories, hard links, dev files, permissions, uid/gid ownership, modification times, extended attributes, acls, and resource forks. Also, rdiff-backup can operate in a bandwidth efficient manner over a pipe, like rsync. Thus you can use rdiff-backup and ssh to securely back a hard drive up to a remote location, and only the differences will be transmitted. Finally, rdiff-backup is easy to use and settings have sensical defaults.

After some tests rdiff-backup appeared to be a great solution, here is how I did a setup with it:

On a storage server I created directories with naming schema: /home/{SERVER_NAME}/{BACKED_DIR}. Then I add crontab lines:

rdiff-backup --remote-schema="ssh %s sudo rdiff-backup --server" backup_user@{server_ip}::/etc /home/{server_name}/etc

Above is an example for /etc backup. The --remote-schema is the command to be executed on source server to backup the directory. I had to change it, due to my sudo setup used for authentication. On every machine I created unprivileged user backup_user and added it to sudoers file without password. I also copied ssh keys to the source machine, so the backup_user could connect without password from backup server. It is not very secure setup, but I manage all servers, there are no external people administrating them and the ssh is secured in a way, that you can only ssh to backup_user on the internal NIC and from the backup server. If You installed rdfif-backup on both machines, the above setup is all you need to backup differentially the machine. I repeated the steps for all machines and directories. After a month I switched off bacula and destroyed the backups. Result? Ive got now 3 months of backup history and 1.3TB free. The data growth is small, due to backing up only changes files. Also I can cd to /home/{server_name}/etc and just see the whole directory structure and copy files with cp command. If I need backup, let's say 10 days old, I can issue rdiff-backup command:

rdiff-backup -r 10D /home/{server_name}/{directory} /home/result

The /home/result will have the 10days old version of directory or file. I can also see the history of backups, using:

rdiff-backup -l /home/{server_name}/{directory}

or use rdiff-backup-statistics command. More usage examples on rdiff-backup homepage.

Summary

Rdiff-backup works in environment since January. It never failed and fulfills all tasks I want. Do not get me wrong, bacula is also a great software, I have got 5+ years experience with it on many production environments, still for such setup as mine with small amount of servers, but still crucial data - rdiff-backup is better. Also company I cooperate with, developed nice and simple web application to monitor backups done with rdiff-backup, we have plans to expand it for management features. Now admins have to add crontab entries by hand, but this is not annoying - you do it once during setup.

If you find this post useful please post a comment, also when you have any criticism. I understand my setup may not be best, so any creative comments will be welcome.

Ubuntu 9.04 jaunty jackalope on dell d420

2009-04-25T20:15:00.002+02:00

Yesterday I upgraded my dell d420 laptop from Ubuntu 8.10 to 9.04. The upgrade I must admit went very smooth, using the update-manager -d method. As a Debian user on server machines for years I was a little bit anxious about GUI method, but hell, world goes forward and I cannot resist.
Here is summary of how is the new Ubuntu working on my precious dell d420 ( btw. a very nice machine ):

Video: work fine with intel driver, I am not a great fan of compiz, so I use the default better appeareance not the additionally installed fusion with plugins etc. Anyways there is one problem, with AccelMethod "UXA", the laptop after suspend shows GDM login screen, so I suspect the Xorg is being restarted. I will file a bug on bugs.ubuntu and see how the guys can help.
Suspend/Hibernate:Works fine, besides the UXA bug. Oh, sometimes, there is no sound after I go back from suspend, the pidgin is blocking the sound card.
WIFI/BT: Works fine as always.
BuiltIn 3g card:Works fine with umstmon and wvdial. The network manager thing is still not working with it fine.
Sound:works
Special buttons:work, the CRT/LCD now started to work properly.
External monitor: this got fixed really. It detects my all external LCDs properly and after detaching the proper resolution is restored. In 8.10 it happened from time to time, that the resolution was restored fine, but there was part of display blacked out.
Touchpad: works

What was additionally fixed - saving of gnome sessions, now this at last works. Also the general responsiveness is better, the system really is faster.

Ubuntu 9.04 released

2009-04-23T14:25:00.002+02:00

Probably You know it, but in case you do not: the new Ubuntu has been released - 9.04 Jaunty Jackalope.

It can be downloaded from : here.

Hurray !!

Working with BibTeX

2009-04-17T12:09:00.000+02:00

BibTeX is a software used for management of the bibliography and references in your texts, usually used with LaTeX documents. BibTeX uses .bib files to store reference info, using a format as follows:

@ARTICLE{DWO09, title = {Title of The Book}, author = { D.Dwornikowski }, year = {2009}, volume = {1}, journal = {Home Journal}, pages = {1-20} }

The above snippet is an example of a BibTeX entry for an article. BibTeX defines some standard classes of documents ex. BOOK, ARTICLE, INPROCEEDINGS, MANUAL, PHDTHESIS, MASTERSTHESIS. They define fields that are required to be filled, so the entry is valid, the snippet above shows required fields for 'Article' entry. The DWO09 field is a key for the entry, it is how the ciation is referenced in the document:

\cite{DWO09}

There is no recommended method of assigning keys to entries, but I use the way specified by our institute:

if there are 3 authors, use first letters of their names and add 2 digit year code: ABC09
if there are more than 4 authors, use first letters of three names and add + and 2 digit year code: ABC+09
if there is one author, use first 3 letters of name and add 2 digit year code: DWO09
if the keys are not unique add letters a,b,c...

You can imagine, that if you have many BibTeX files, containing hundreds of entries, the management of them is becoming simply a pain. However there are are couple of usable applications, that can help you with your collections of bibs. Personally I use JabRef, which is multi platform java program with many powerful features, like LyX integration, online search of citations and powerful bib management engine. There is also OSX application called BibDesk, Gnome Referencer and KBibTeX (try to pronouce it .. ) for KDE.

PS: When talking about BibTeX, I have to mention online citation databases, where you can find publications and metadata for bib files ( some of them even generate ready to use BibTeX entries ).

LyX TIP: figures and pictures placement in text

2009-04-16T15:23:00.003+02:00

A small TIP for all LyX users. While working with LyX, you often place pictures, diagrams and graphs into your document. The default behavior of a picture float field is such, that in most cases text, which in LyX is before the float field, in the PDF appears after it. This is however a normal behavior of float, even though we do not want it. Here is a TIP on proper and nice placing of figures in LyX.

1. You place the Figure Wrap Float.
2. Click on the settings of the float and switch on advanced placement settings:
top of page
bottom of page
here if possible
3. Insert image into float
4. Click on image properties
5. Put 100 in Set width and choose "Text Width %"

A good manner is also to put a label just before the description in the figure wrap float.
An example:

Click to see bigger:

My subjective view of a word processor idea

2009-04-14T14:09:00.007+02:00

Introduction

I decided to write this blog entry to show my subjective view on editing and publishing documents using nowadays computer tools. It is also a blog entry showing that idea of what we know as word processor is wrong and no really useful. People just do not know nothing else. This post is a product of my recent work as an academic researcher, where my job is generally to write A LOT. I would like to share some thoughts and experience on the topic of documents writing and tools used for it.

Why is word processor a bad idea?

Word processor is a computer application, that generally lets you edit and create text documents. This fact is commonly known. Probably it is one of the mostly used computer applications in the world, as the history of computer - as an office tool has its direct origin in typewriters. People just switched from these mechanical devices to PCs with printers, as a more convenient and effective way of working with texts. Word processors evolved from very clumsy and poor featured notepad like applications to feature rich programs, parts of whole office suites. What did not evolve with them was the main idea of word processor - and when I say "word processor" I think of MS Word/OO Writer like application.

Not taking under consideration the templating abilities of Word/Writer applications, generally word processors are 'WYSIWYG' ( What You See Is What You Get ) programs, this means that the document looks exactly the same as when edited. For most of people this can be an advantage, but if you think more about it this is a great shortcoming. Imagine you create a publication, let's say 200 pages big, using your own style. After some time, the template changes, ex. your company, school - whatever changed drastically the preferred layout. Everybody, who used Word, can problably feel the pain of copying and adjusting the text to the new template. This is due to not really usable or non-existent template system in word processors.

Another annoying thing, and worth rethinking is collaborative work on documents, some time ago I posted about a nice method to solve that in LyX/LateX environments. Modern word processors, due to the WYSIWYG feature are just not usable for collaborative work. You have common format of document ( ex. DOCX, ODT ) and theoretically it should look the same on all suites supporting them, but in most cases it does not. You end up on doing few hour lasting merge on one machine to have a consistent and nice looking version.

The last most annoying thing in word processors is that the document is treated as one big stripe ( toilet paper ). When you add something the the beggining, the rest moves down. After that you have to go through the whole document and make changes, so you do not end up with blank pages, figures without descriptions and garbled tables. This is the feature inherited directly from typewriters, top bottom approach in editing texts.

What is the answer to that ?

In my opinion the idea of word processors should be dramatically rethought and redesigned. Instead of having non usable WYSIWYG applications, we could work on texts using the WYSIWYM ( What You See Is What You Meann ) idea, you just write what you 'mean' and the templating systems makes it look how you defined it. This is how LaTeX/LyX work and more or less the Pages application from Apple iWork. My intention is not to force to use LaTeX of course, I just really thing the pure idea of separation between data and presentation is great. We use it in programming, we use it in www, we use it almost everywhere, who not to use in publishing. The change would not be drastic, even transparent.
Let's take LyX as example - being a graphical application, it still needs some LaTeX knowledge from the editor, but I think there is no limitation in creating an application, that will manage to present changes in on-line matter - you edit and instantly see the output document. ( in LyX/LaTeX environments, you have to clearly initiate the action of compiling to ex. PDF format ).

I am a surprised by the fact that during last 20 years of home/office computing none of leading companies producing office suites ( Microsoft, Lotus, Sun ) did not try to change the main idea behind the word processors, the change would not affect drastically the users of the applications, and frankly they would not probably have much to say ( Microsoft has 90% of office suites market ).

Any comments appreciated. Thanks for reading.

WS-* - would too many standards make WS end up like CORBA?

2009-04-08T15:53:00.003+02:00

Service Oriented Architecture

SOA - Service oriented architecture, a new and very trendy paradigm for building distributed IT systems is having it's 5 minutes in the enterprise IT world. Big software vendors like Microsoft, BEA, Sun and IBM, as well as many consulting companies develop more and more tools and platforms for SOA deployment. For most people, even those, who know more about service oriented architecture, the SOA term is associated only with Web Services and SOAP ( Simple Object Access Protocol ), they do not know that REST, CORBA, DCOM , RMI or SunRPC can be also referred as SOA. I am not surprised though. While undertaking my research on SOA systems monitoring and management, I also got the impression, that everything that the industry is caring about is Web Services model. Just take a look at OASIS, the main standardization company for WS-* family and you will understand what I mean. There are plenty of different standards describing and trying to normalize WSs in heterogeneous systems. All for our good, but the question is : is it good for WSs itself?

SOA ? You mean SOAP and web services?

A short CORBA story

Back in the old days, when vendors used to be compatible only with their own standard, there came OMG ( Object Management Group ) and created a specification for what industry took as a drug for their problems - CORBA ( Common Object Request Broker Architecture ). CORBA was meant to be the glue, that combines various technologically different systems, using one concise and common interface, by standardization the way objects are described, serialized and transferred between systems. The idea was great, suddenly a great boom went off among vendors, clients and integrators. Many companies started to produce their CORBA implementations and ... it was the beginning of the end of poor CORBA. The problem was, that standardization was open, both software vendors and software users could participate in the process of standard creation and further modification. Of course both of these groups did not share common interests. Vendors did not want to change drastically their existing implementations, so they were trying to block major modifications in the standard, users wanted to ease development and opted for progress. The whole process was open, what according to Michi Henning in his article published in ACM Queue titled "The Rise and Fall of CORBA" was one of the major factors that made CORBA disappear.
The second one, in my opinion, was the sophistication. Everyone who even tried to implement a simple CORBA application probably spent a few hours on simply trying to understand what is going on, not mentioning the complexity of the code. The last significant CORBA killer factor was that CORBA vendors used to choose from the specification parts they wanted to implement, omitting the problematic ones. Lack of some certification of such implementations led to the situation, where two different implementations could not connect to each other.... and we are talking about technology that was meant to be the cure for this problem...

Web Services, OASIS and the future

Anyone, who is into the Web Services technology is probably familiar with OASIS ( Organization for the Advancement of Structured Information Standards ), the consortium responsible for standardization of WS-*. Not everybody though, especially not developers, who use RAD tools, is familiar with WS standards, what is generally a very healthy approach.
During my scientific work on different SOA environments I am obliged to understand the core of the problem, what in terms of WSs means reading and analyzing plenty of standards published under OASIS and other organizations, like ex. DMTF ( Distributed Management Task Force ). Im a big fan of SOA paradigm and distributed systems in general, as I can see it as the future of computing, but cannot loose the impression, that WS-* are starting to become another CORBA. Of course Web Services are far better designed and implemented, than the Common Object Request Broker Architecture, most of the shortcomings of the latter are not simply present here, but still the "Keep It Simple Stupid" philosophy does not apply here. Too many optional standards is a threat to compatibility, and when this one is lost, WSs will start to lack its major advances - the interoperability, the scalability and the heterogeneous architecture. Let's hope vendors will try to keep their products compatible with each other and implement most of the standards. I might have got a wrong impression about the WS-*, so any comments are welcome :)

Disclaimers:
The image is from http://csiwebservices.net/Pages/Default.aspx page.
The link to the ACM article may not work for non ACM members.

FON a free wifi access

2009-04-06T13:48:00.000+02:00

Today while browsing engadget in my google reader feeds, I encountered an article about new FON wifi access point. I am generally into networking, so I decided to read and know more about FON, the company providing free wifi access to the world.

How does FON work?

With FON you can get free wifi ( or paid ) access wherever you see FON wifi around you. It bases on community of people willing to share part of their home internet connection to the surrounding. You can become a FON hotspot by buying the FON creafted access point or upgrading your Linksys firware. You also have to register to the site. What do you get in return? Undoubtly a joy of being a community member and a free hotspot, but also free access for yourself to other FON hotspots. Also you can sell your internet connection, but this seems to be non profitable ( the FON site says users make 3 EURO a month on average ).

FON has three classes of members:
- Foneros - those who give internet access for free
- Aliens - those who use internet access for free
- Bills - those who sell internet access

You can check FON hotspots around you using google maps.

Collaborative work on documents

2009-04-03T10:10:00.000+02:00

Scientific work in most cases is collaborative, that means more than one person creates and edits the document. In case when this involves only two editors, the problem is not so big, but when you have for example four or five of them, working on one document is becoming simply hell.

Where normally people use Word or OO Writer, in such cases these types of programs are simply not usable at all. The .doc format or .odt even is not a real problem, but the idea of word processor alone is not very successful on the fields of scientific publications. From my experience, working on one Word compatible document collaboratively means continuous emailing and merging changes to have a valid working copy.
Fortunately there is LaTeX, commonly used by academic societies for writing publications. For those who do not know what LaTeX is, wiki comes for help:

LaTeX is most widely used by mathematicians, scientists, engineers, philosophers, scholars in academia and the commercial world, and other professionals.^[1] As a primary or intermediate format (e.g. translating DocBook and other XML-based formats to PDF), LaTeX is used because of the high quality of typesetting achievable by TeX. The typesetting system offers programmable desktop publishing features and extensive facilities for automating most aspects of typesetting and desktop publishing, including numbering and cross-referencing, tables and figures, page layout and bibliographies.

The biggest LaTeX/TeX advantage is that you focus on what you write, not on how it looks like, LaTeX styles and document classes define, how will your paper look like. You can also define your own styles, like article, book, etc. Also export to known formats, that look on every system the same is a powerful feature of TeX/LaTeX, you can export your work to PDF and be sure that people will see it the same ( taking under consideration, that they have fonts installed ).

However writing in LaTeX is not so easy, for regular user it reminds programming, and basically it is. That is why there are many helping applications, but in my opinion the best of them is LyX, the GUI editor for LaTeX. It works on all of commonly used systems like Linux, Unix family, Windows and OSX. Under the nice GUI LyX format is a plain text file with more-or-less LaTeX like syntax. Why I mention this ? This is the reason it is easy to work collaboratively on LyX documents, using LyX built-in change tracking mechanism, or an external version control system like Subversion or GIT. This is in fact the method we use in out institute. Every publication has its own svn repository, where LyX files are stored. Working on one copy is just as easy as update/commit, and all changes are merged easily, as we still work on plain text underneath.

Additionally if you use BibTeX, system for bibliografphy handling in LaTeX like systems, you can have separate repository used by everyone with bib databases to include easily in your publications.

Summing up, I encourage everyone to use LaTeX/LyX for their publications, not only if You are a scientists, but in a normal regular work too.

Links:

LyX homepage
LaTeX homepage

Ubuntu feelings

2009-04-02T15:24:00.000+02:00

I was always linux enthusiast, but for last two years I used OSX on macbook pro. It is a nice combo, until it breaks. I went back to origins and bought dell d430 laptop and tried ubuntu for the first time. I was always a debian user, so ubuntu was more or less natural choice, as it strongly bases on it. My impression is simpy "wow". Everything works just out of the box on my dell, even the HDSPA built-in card from vodafone. After my 2 years absense in the topic of 'linux-on-desktop' I am simply shocked by what Canonical achieved with Ubu. I also stared to use gnome, and find it very usable ( I was always a KDE user ), but that does not mean I won't switch to kde 4 in future when it gets more stable. Now it is time when new ubuntu 9.04 goes live on 25th of april. I look forward especially to ext4 filesystem and new gnome, which improves some things I use. Of course there are some things that linux desktop lacks, and one of them is no nice IM application, like adiumX on OSX. Pidgin is nice, but... could be nicer. Im not a very demanding user, I generally need terminal, LyX, open office 3 , firefox and pidgin for normal work, the rest is just addition.

PS. a nice thing i found in gnome is tracker, which is the answer for my needs of indexing plenty of sciencie publications in PDF. With tracker I can easily find an article, from which I remember only one word. Good job.

New life, new blog

2009-04-02T14:24:00.001+02:00

First post on my new blog, time to introduce myself. My name is Dariusz Dwornikowski and I live and work in Poznań beautiful city in western Poland. Recently I drastically changed my life, dropped business and decided to dedicate myself to science and go to back to Poznań University of Technology to work in Computer Science Institute as a researcher of SOA technologies. From september 2009 I am also starting offcially my PHD studies. Im thinking about thesis about distributed management of RESTful web services in SOA Grids, but that can change. Im also a great open source and linux enthusiast, using it in every day life. My interests besides distributed systems are networking, IT security, voip and unix operating systems. In my spare time I play table football and watch movies.