StoneFire Development

Crashing iPhone

2008-08-04T17:54:00.003+02:00

Well. The iPhone wont last forever. Last week I has some weird problem with my iPhone, and started to wonder if it was due time for a service. After 2 months of use... That's bad. The problem I had in the beginning was a somewhat unresponsive touch display. I had to press and drag multiple times before it reacted, and it had unexpected restarts. I didn't really think much of it until on my way home, I decided to change play list, and took my iPhone out of my pocket. And it immediately crashed. At the same time it felt blazing hot, so I thought maybe the hot weather accompanied by transportation in my jeans pocket in a hard plastic casing became too much insulation. E.g. overheating. So I set it for cooling, and started it up again later that evening. And so it worked after all fine the next day or so. But the next day, I experienced series of unexpected crashed, often when I was "lighting" the display to see what the current song's record was (need to learn which version of each song belongs to which record you know...). And the same moment I pressed the "home" button, it crashed. And this happened twice, plus one or two unprovoked crashes. Later on the day, I met with some friends, and used the Clock program as a pizza timer. But after 2 minutes, it crashed. This time it really started to act weird... The screen started to blink, and was blinking black and white at about 60 bpm. And it didn't respond to any controls. Not even the "sleep" button. After a while I would kill it, and it returned to showing the apple logo. First the ordinary splash screen. Then it was tinted red, then it crashed again. Oh god, here we go. And I pressed everything on the phone until it switched off. And then the pizza got a little burned. OK I thought... It was playing the hard card now. It needs service. But hey, the Norwegian iPhones are not sold though the Apple stores, but through the iPhone partner stores: Who sells it with subscription bundled. You actually need to sign the subscription to get the phone. But I had bought it in an Apple Store. And the closest "Apple Store" with iPhone to Trondheim which I could use? New York, in terms of travel time. So, OK. I need to figure how to maintain this thing myself, and searched for a lot of materials on crashing iPhones. OK, Apple don't seem to acknowledge that the iPhone can crash, but there were a few videos on Youtube, including this one, which shows almost all the problems I had. On one of the videos I found, it looked like the person holding the phone squeezed in a way. So I tried that. And then I tried to "twist" it a little. Instant crash. Then twisting it back: meaning while holding the iPhone with the "home" button to the left, I twisted the "left back" and the "right front" down while the two other corners up. Which fixed it instantly. The opposite twist seemed to instantly crashing it causing the "blinking screen" problem to appear. So I twisted it the "right" way a little, and squeezed it together, and since then I have had no problem at all... My guess is it has some loose contacts with the touch screen, and bad software for handling the faulty connection. It also seemed to colorize the screen in a red tinting (meaning it was unable to turn "off" the red pixels), a the same time it lost contact with the touch screen. But I have no real way of thecking this without ruining warranties etc. I have a 16G iPhone with 1.1.4 software.

Protocol Buffers

2008-07-10T23:24:00.000+02:00

Funny date to be announced, but never the less. On my 29th birthday, Google announced Open-Sourcing the almost ubiquitous Protocol Buffers. Protocol Buffers is simply put a way to store data, and to convert this data between internal data structures, which in Google´s sense must be extremely optimized, to string data like readable files, RPC buffers, etc. And we need it in C++, Python, Java and more. In daily "programming" life, Protocol Buffers are primarily used as extensible and easy to debug data structures. Since we can convert the data. Any kind of data into files in a whim, it´s easy to test any kind of operation on it. E.g. we have a model:

message SearchReqest {
  required uint64 request_id = 1;  // resuest message id use for message tracking.
  optional uint64 requestor_id = 2;  // requestor id is the HashTable ID of the one who want´s the answer.
  required uint64 searching_for_id = 3;  // we want to find this id.
}

message SearchResponse {
  required uint64 request_id;  // Same as in SearchRequest.
  optional uint32 ip_of_owner;  // int encoded IPv4 f the owner IF FOUND.
  optional string error_msg;  // error message if it failed.
}

(Example is taken from my Master Thesis, Titled "Consistent Lookup in Distributed Hash Tables") In this case we can make a hash table given a node setup, and just make a large set of search requests, some node setup data (e.g. node connections, node status etc.), and expected response output. Perfect for debugging, and already contains methods for protocol transfer (ASCII variants, binary formats etc.) If you are interested, go to http://code.google.com/p/protobuf/ and take a look.

OS X Improvemens #2 - Contacts and Calendar

2008-04-09T10:23:00.000+02:00

As a second look at Mac OS X, I'll look at the Address book, and Calendar applications.

The address book and the Calendar are small and handy applications. Their meant to be synced against the .Mac service, which is an online services center provided by Apple. It has mail, address synchronization, calendar, and is connected to the iTunes music store. But hey, here starts the problems. The calendar and address book seems to almost only meant to be synchronized toward the .Mac... But it has LDAP support, and import and export functionality, but no built-in support (or seems like possibility) to sync to other such services. (Although iCal subscription support in Calendar).

[feature/bug type, importance/impact, cost]

Address Book

Just to look at the needed changes in Address book:

[usability, low, low] To be able to see group belonging when watching a contact entry. Like when looking at "Ola Normann", I would like to see a "Groups: Work" etc., or if contact is not in any group, no such marking. Implementation: Add a reverse group mapping on from each contact object to the groups objects [e.g. hash_set in_groups]. The mapping can be done on the fly when the contacts are loaded, and make no change to the storage or any other part of the interface.
[usability, high, low] Smart groups is missing a vital option; to be able to use group belonging as criteria for making a smart group. I could make "union groups" or "negative groups", like; "All contacts in friends, fencing and biking", or (the reason I discovered the need); "All contacts not in any group", or "Contacts in friends, but not in fencing". This can be implemented by adding a filter on the reverse group mapping as mentioned above. And adding a wildcard or option for "Any Group".

This is quite few issues for such a program, but hey, their important to me.

Calendar

The calendar is quite good, but hey, there's still issues... But well. To be honest, I don't use the calendar app on the Mac except for subscribing to my Google Calendar, so it will be synchronized to my iPhone. But I would really like a plug-in to synchronize (two-way) between my Google Calendar and iCal. How to do this, and how much work it would make is uncertain, but I'd like it still. Maybe I even could start to use it at all...

iPhone

The iPhone isn't realy any of these applications, but it interacts a lot with them. And there are still issues with this. These are mostly issues with the iPhone variants of the same apps, and not with the interaction itself, which works just fine for me.

[usability, high, low] Contacts creation has a problem with when to save it to the address book. When I make a contact, then add a name and company, and press "save" (and goes back to view the contact), I expect the contact to be saved too, as when I edit a contact. I too often makes this mistake, and looses contact information because I don't press "save" in the "view contact" window, but use the "home" button. Implementation: Add a check if contact has been created on saving single contact item. It will require a little more disk use / power, but should be trivial. And contact updating is not that frequent...

OS X Improvemens #1

2008-03-23T18:11:00.004+01:00

OK, I'll stop ranting about OS X now, and rather think constructively. It's no help in having yet another twit barging about whats wrong with an OS without trying to solve the problem too. I will try to make a list of improvements to OS X and its default applications that would make me more comfortable to recommend Mac to more inexperienced users instead of Windows. Which I really would like to, because it's a better OS in so many ways, but not in too many others.

I think I'll make this a series of posts and not one gargantuan, and rather take one application at a time. And naturally I will start with iTunes. I chose iTunes because I have the latest version, and is quite sure my problems are not fixed in a release until a few weeks ago. On later posts, I may talk about applications in non-most-recent versions of OS X, as I still use Tiger...

I have iTunes 7.6.1 (9), running on OS X 10.4.7 Tiger.

But iTunes. iTunes is a multimedia application trying to merge all listening of music, watching of music videos, online TV shows and radio via podcasts, and movies. Quite a lot to strive for of one application, but they have done a quite nice job there. I will propose some improvements on default behavior, some may which have been solved in plugins or secondary apps. I want this in the default iTunes.

Improvement #1: Artwork
In my opinion, album artwork should be available without an iTunes stores account.
Improvement #2: Multiple Collections
Since I have a rather large music collection of many artists and albums, I would like to separate my collections. I have a removable HD of 500 MB where most of the music lies, and music videos, movies etc. And a way to "annotate" a collection with what it contains. E.G. my video collection would be imported as "Music Videos" not "Movies".
- Take inspiration from Amarok. They have done a great work with collection, although as s single one.
Improvement #3: Collection
File Management When managing files in a collection, I would like to set how to manage my files ad folders. E.G. I want to a) Keep files within same collection, no move to central repository. b) Be able to set folder structure manually, e.g.
{$COLLECITON}/{$ARTIST|limit:1}/{$ARTIST}/{$ALBUM}/{$TRACK - }{$TITLE}.mp3.
Improvement #4: Ringtones
I see no reason why ringtones (for my iPhone) must have been bought in iTunes store. As a member of a musical family, I might want self made music into ringtones, or to convert from a rare CD (or even LP) i have to ringtones.
Improvement #5: Music File Format
There are many file formats out there. And I miss native OGG and FLAC support for iTunes (and iPod and iPhone). Another good thing would be DivX, and DVD .VOB. Secondary fix would be to change the quite disappointing "File is not a video file." error message into "File is not in a supported video format.". At least when I have another program that reads the file without any problems (see MPlayer). The same goes for Quicktime Player. I actually feel ashamed on behalf of Apple, when the renegade open source applications beat the heck out of Apple in multimedia file support...
Improvement #6: Album View
Album view of a series of "unknown" albums are separated into each "album", which takes much more visible space than anthing else. I propose to treat "unknown album" as the same album for a given artist. So instead of seeing two albums with songs, and five "unknown albums" with one song each, merge those "unknown" to one.
Improvement #7: Missing Files Detection
If in an online collection, a file is missing, iTunes should be able to recognize that as that file has been removed, and should be removed from the cached collection.

That is all I have for now. All in all, iTunes is a good application, but with these changes, I would really start to like it too, and stop irritating myself over what I feel is lacking. I thought of writing about iPhoto next time, but that has been removed from the default installation of OS X. But I will find another application. On the list is:

Finder and file management.
Preview and image collections on Mac.
Calendar and Address Book management.
iPhone and iPod Touch interaction with Mac and the Internet.

A Fair Look At Mac

2008-03-21T00:06:00.003+01:00

I have had some time with mac now. And just once again, I get a lot of small irritating problems. In general I'm satisfied with it, but good gracious. Apple need to get their act together if this is going to be my platform in the future too. Why? Because despite the shiny interface. It has some trmendously irritating "features" I dislike more than I can "like" the interface.

Problem #1: Keeping things updated. I'm not talking about "up to date software". But just keeping programs aware of changes in files or folders in the background and act uppon that. It's quite irritating to have to manage my music through iTunes, just because iTunes can't handle changes in files and folders. If I clean up available music I have to do that through iTunes, or it fails to see that anything have happened with the music files. Same problem is with the Finder (file manager of Mac), which caches each folder's content, and don't even have a refresh function to update what's there.
The solution to this is not big at all. All other OS'es I have used can see that. A folder is changed, so update it's content! And add a "refresh" function to folders in Finder.

Problem #2: Support of music and video formats is horrendous on mac. Really of no other reason that many mpeg and avi variants can't be read by Quicktime or iTunes. Same with OGG Vorbis. Even if Apple have their own music file format doesn't mean it's better than any other format. I like OGG, and I have loads of AVI and MPEG videos that my apple programs can't handle.

Problem #3: iTunes... I should'nt need to say more. It's quite a nice program, but with some flaws that I don't handle wery well. I tunes should have more functions for changing file information and grouping. Better discovery of duplicate files, file updates etc. Well, I'll just make you a list.

When "adding files" to iTunes, all video files are assumed to be movies, but in my case their mostly music videos. And there's no shortcut for changing a range of movies to music videos. Make one!
I have quite a lot of both music and music videos. I'd like those to be separated. Also in separate folders when automatically arranging folders. And for gods sake, support the video formats!
OGG Vorbis isn't supported. That's a flaw in itself.
When updating meta information on songs. Please update the files too. It's irritating when I make such updates on names etc, then copy to my portable HD, import on a different computer, on which I have to do the same meta updates.

Problem #4: iPhoto. Since iPhoto is the default photo managing program in Mac, it will get the same scrutiny as I would give any other OS. And the problem of Mac, is that the "Prewiev" program simply sucks. I cant just open a folder in preview, and browse the pictures as a slide show. I must have iPhoto. And iPhoto is terrible at managing images when you pass over a few houndred of them. And my collection of 6500 pictures (self taken), is small compared to others.

All in all, what I'm disappointed in is the need for buying more software to be able to do ordinary tasks, tasks that I expected Apple to know about and solve, and include in the operating system. And some of the software lacks functionality I recon as essential to it's use. And many Open Source programs solve exactly those problems. Except they dont work as well on mac. Maybe Linux still is my OS after all.

Maybe it wasn't as fair then, but my view anyway.

MVC and RBAC

2007-12-16T23:46:00.000+01:00

MVC is a relative new term in web development. The whole MVC philosophy is on how to organize code on the server (PHP, Ruby, JSP etc.), and how to separate the various function in development. A short summary of MVC is to separate server code in three groups.

MODEL: A layer to manage data retrieval and storage. Keeping data structures and the direct modification of these.
VIEW: A layer for generating visual data. That is generating HTML, Scripts, XML and the like of some predefined data structure.
CONTROLLER: The connecting structure between the HTTP call, the data and generating views. A "perfect" controller function has three steps, (1) deciding what data to load. (2) Load them, and (3) show them.

And added to that we also have:

LIBRARIES: Global classes and function library where we put what's not really belongs any of the above groups. Can be data caches, access control, translation system etc.

I still don't have much experience with MVC, but what I've used so far (code igniter, and a touch if RoR and Zend), but the difference from programming all in one block is huge. My last project in Market Monitor is to make a dynamic access control system fit for our internal structure, which became a full fledged RBAC (Role Based Access Control) system. And it wasn't really difficult either. Using the MVC structure, it was easy to separate the various functions, and to build a set of library classes to manage users, roles and resources. In addition to the groups and the ACL. This contains: Database relations: For the time being, using the MySQL database, but should be programmed in a general SQL97 syntax, so it'll be DB independent.

auth_roles: Roles overview.
auth_subroles: Roles in role graph. Note: This must NOT become circular.
auth_groups: Group overview.
auth_grouproles: Groups can have roles associated with them.
auth_users: Users, passwords and user information.
auth_userroles: Which users have which roles (directly)
auth_resources: Resources (with hierarchy)
auth_acl: The Access Control List (user or role to allow or deny).

And a model class (UserDAO for loading and storing the data here, should possibly be renamed "AuthDAO"). And library classes:

Role: A simple role container.
User: A user container. Changes some behaviour regarding users, and separated users from roles (needed in the authorization process to prevent too much recursion).
Resource: The resource tree, and with the core authorize method.
Auth: Library class for storing authentication data and methods. Simplifies most authorization to a single line: if($auth->authorize("$resource_path")). The only requirement, loading an instance of Auth, and having a login possibility.

And most important of all... A controller for administering of the authentication system. Managing roles, groups, users and resources, dependent of the admin's rights. Be it system administrator or some group administrator. This system implements in total a complete Role Based Access Control system, and with relative little code. Some 2000 lines in total, spread on 6 classes, and some utility functions. Too bad I'm leaving that workplace. Would really like to see how this system is utilized, and to be allowed to work on it further.

My AJAX and Web development Tools

2007-12-10T21:09:00.000+01:00

After now almost 2 years in a job with extensive web development, I have also come quite into using AJAX as a mean of increasing usability. But is it really feasible, and what is it really good for in the end? Earlier I have been using prototype, which was good, but has some not so good flaws... The bad sides about prototype becomes aparent in two situations. Firstly in long time use, where the user has a single page open which does repetitive "live" updates. But prototype is designed to load HTML pages, and put it in "containers" in the live web page. This gives a bad problem especially in Firefox, which caches all these pages until the entire page is reloaded or closed. The other problem is also visible in other browsers, but is not as severe. Prototype "extends" every DOM object in the page that it works on with a large set of new data and methods. And in cases of large pages, this can become an extensively large payload. And this "attribute"in combination with the previous can put the fox to a complete halt, even on a powerful workstation. And the solution? jQuery! jQuery is a javascript library compatible with all the major browsers (IE6+, FF 1.5+, Opera 8.5+ and Safari (on which I don't know the required version)). For a list of the differences between jQuery and prototype, look further down. But the major point is a design difference. jQuery (jQ) is build like a "standalone" library that tries to change as little as possible to the DOM structure. All "new" objects and methods are separate of the DOM, and will rather contain a set of DOM objects, than extending themselves. But in more detail.

Prototype uses a set of defined OO function extending the DOM tree itself as the base of the library. jQ only creates the jQuery function (and object and methods, and the '$' shorthand, which sadly collides with the '$' method in prototype).
Prototype works on a single DOM object at a time, while jQ can collect DOM objects, and work on them with callback functions.
Prototype is bound to work from the ID's of the DOM objects, and otherwise do manual programming in JS. jQ can access the entire DOM tree through both XPATH and CSS like syntax.
Prototype uses a set of extended JS collection classes to administer data, while jQ uses simple JSON structues and Arrays, and has internal methods for iterating them.
Prototype has no "stable" DOM builder (as of last I checked, a few weeks ago), jQ has a simple extension: jQuery-DOM that does the job, almost too easily.

I guess I could go on like a real rant, but what the heck. I'm hopefully not going to work that much with AJAX and JavaScript in the future, as I am changing employer in just a few weeks. Other web programming tools that have come before me since last post here, is the Code Igniter MVC framework for PHP. As opposed to Zend, which I have looked at earlier, Code Igniter (CI) is not dependent on PHP5, but also works fine on PHP4. It is also cleaner, in the sense, that it has less "optional/extension" code that most users wont need. And a good property of CI, is the BSD license, which makes it easy to take parts of the system, and integrate it into your business' code. Both CI and MVC is good, but I wont take any of those into details now. But take a look if you struggle with your PHP code, and want a system to clean up out code.

Linux in a Windows Environment

2007-03-16T22:05:00.000+01:00

I have been using Linux at work for more than half a year. And here is my experience so far. It's mostly only positive, but I have had some problems now and then, and they relate to two major problems: Old and outdated programs in use, and collaborating with Windows users, windows developers, and designing with Windows platform support. There are a few tools that have made the job quite easy, like remote desktop support, MSN, Eclipse, MySQL Query Browser for Linux, Firefox, and IEs4Linux. The last one have enabled installment of Internet Explorer on Linux, making it possible to test pages on IE (with flash!) without changing back to Windows. I have also installed MySQL and Apache on my workstation, and I have a multitude of tools to help me here and there. We are four Linux-users on workstations now, of a total of 17 workers. The rest is using Windows XP as primary OS, and one is using a Mac laptop as helper computer (... yes, our designer). But Linux is spreading slowly through our system. And I think it's thanks to us using Linux, and showing the others that it's not only possible, but its reliable, fast and on almost all cases, just as good or better than Windows. The downfall for most of the others is the lack of windows software, but that is solved though series of similar or compatible software, web applications, and trans-platform applications. The list isn't long, but here it is. I earlier made a list for Windows, but now comes the list for Linux:

Eclipse:
Not much has changed about Eclipse. It's been removed a couple of bugs, and the Java version has been upgraded to 6.0 (instead of Java 2 SE 5.0, having JRE version 1.5.0 ...), which made it a teeny bit faster. The subversion support is a little better, and the HTML and XML tools are a little better. PHPEclipse is the same old, but it hasn't had any significant development for over a year.
Firefox:
Once again, the same tool as on windows. But now with a tighter and more throught through set of Add-Ons. In addition to those mentioned earlier: I have tamper data, faviconize tabs, aging tabs and tab control.
MySQL Query Browser:
Still the same, but a little buggy on Linux, so the functionality is a little less. I cannot use the built in table edit methods, but that doesn't matter much, I have other tools to use.
Gnome Console:
With a powerful console with the wide variety of console tools that is available on any standard Linux computer, I can do almost everything. From nano, through svn, grep and sed, vim, a full LAMP server locally, the console gives me a power I never had when I was developing on Windows. Even though what I got with Cygwin gave me a taste of console goodies, I still prefer to work on a «real» desktop with a «real» console...
IEs4Linux
Even just a simple wrapper around wine with IE 6, it makes it able for me to check web pages in IE 6, even though Linux don't support IE ... Or does it? I guess Bill Gates don't like this hack much, but I guess there's little he can do. He has made his own software available for free...

This is the basic list, and a lot of other tools are used, but mostly on the console. The availability for powerful console tools on Linux is immense, and the power I as a developer gets is worth the hassle I get when developing web-pages which have to support the various versions of IE. My prefferred desktop distribution is Ubuntu. The Swahili named distro has taken great care in making the installation simple, fast and even possible for a Windows user to understand. And this despite not going on the «make it like windows» style that Linspire has done. Ubuntu is a user friendly Linux distro, and has made it into every computer I have. And thanks to this my workstation haven't seen (or heard of) Windows for more than half a year now. And the box is one of the most reliable computers on work. Even the sysadmin looks over my shoulder once in a while and seem to want to have a Linux box himself, although he haven't said it out loud yet.

Release!

2007-02-27T22:38:00.000+01:00

Final release had come. Tomorrow is officially the last day we have to work on the new «Betradar 2» web system, combining a ton of new features we have been working on for quite a while. I myself has worked on new statistics features, and it is finally finished. The last round of testing is over, and now I have more or less nothing more to do on it. The last hustles were the merging process which I have used the last two days on. The one special feature I have been doing is the Live League Tables. But before I could make the live tables, I had to make a new controller for making «normal» league tables. This time static tables that are not calculated on the fly, but calculated as matches are finished, and stored in its own database tables. The idea is simple. Instead of finding out how the table should look when a customer wants to look at it, then it's calculated in advance, and stored. The load on the web server is quite diminished, and there is a wider possibility for customizing the tables in advanced ways. Adding more complex information to the tables are also quite suddenly possible. Like partial statistics (Last 10 matches), comparable statistics (GB), or building complex table structures (NBA, NFL).

As simple as it sounds, it took me months to finish. And I used several rounds of programming, searching and failing before I came to a solution which actually worked perfectly. The few breakthroughs where all linked to applying frameworks of some sort to automate some part of the development, or make easier to control the enormous information flow. In total I ended up in creating 7 new tables in the database, although making some other obsolete. And as one day an old controller died that had taken care of maintaining some of the «old» tables, and I found myself unable to find and fix the bug, I simply made an extension to the new controller that «re-built» the same information (from the new tables), and put them into the old one's, which contained more or less the same information. One of the jobs I have ahead now is to try to phase out the uses of the old static systems, which is controlled by code that is barely maintainable, and the one who created it has quit, so when it needs some fixes, we have big problems just finding out what to make of it before one of the «older» guys takes care of the problem. Its tedious and quite irritating to be frank.

Sadly I cannot publish the link to the new web site yet, but I will when it goes public!

My Desktop Tools

2007-02-22T21:56:00.000+01:00

All the half serious linux news sites and bloggs have posted about what tools they use on their desktop. For me, its going to be a little more complex, since I have 3 desktops to manage, and they are quite different. But first I can introduce you to my 3 computers (although one belongs to Market Monitor, its still one of my desktops). [Gwynn], a Fujitsu-Siemens T3010 bought in november 2004. I found out that since they had a new model out with more features, the old one came much cheaper. It has a Celeron Mobile 1200 MHz CPU and 768 MB RAM (shared with video card), and a 40 GB HardDrive. The laptop is also a tabletop and has a stylus... Very handy for drawing, and a 12.1'' LCD TFT Screen. [Oasis], an Athlon XP 2200+ (1.8 GHz) with 1 GB RAM, and 60+80 GB HardDrives. It has an 18 inch Samsung 181T LCD TFT Screen, and some cheap phillips speakers. Based partly on my old computer; including the screen, one of the harddrives and the cabinet, Oasis is the oldest computer I have. The screen is actually some of the oldest there, as I bought it in january 2002 for money I had earned the previous semester. [Kada], a Pentium 4 3.8 GHz with 1 GB RAM and 80 GB HardDrive. It has 2× 19 inch LCD monitors, and a DVD burner. If it's not already clear; this is my desktop at work. Strange enough, as an old Linux fantast, I have Windows XP most of the computers. For my two home desktops, they have each different reason. The laptop does actually have no CD-ROM, and is therefor 100% dependent on network for everything ... even installing Linux. That was a fun project, but I will not go into that now. And to have something that just works on the computer I take with me is really great. It took several months before I got the time and effort to install linux on Gwynn, and after that it still took months until I got it to work just right. But Ubuntu has been a walk on roses. I don't know how much it has hepled me, but I shifted the distribution on all of my computers in just a short period after I installed Ubontu on Kada. I also have an old computers that I really thought I had gotten rid of. [Zoe], a Celeron 400 with 192 MB RAM, and a mere 4.7 GB Hard Drive. It has a 16 bit 800x600 screen LCD monitor at 10.2 inches, although there is an enormous rim around it... It is the old laptop, which is even too old to run any modern serious applications. But since it's quiet and still works quite good, I have spared it for the possible use for small IRC session computer, or just for play. But I haven't had much time for it, so now it has no working OS on, although I want to get it an Ubuntu-Server installation or something. But that's for the future. All the names and the illustrations are taken from the female characters of Sluggy Freelance, a network cartoon I have followed for almost three years now, and it has been running for many more. Some of the computers are older, and have had other named before, like [Oasis] was called [Tienbao], and i have given away a computer I called [Jinbao].

The Stage Machine

2006-08-09T21:18:00.000+02:00

Most people with some programming experience have come to know the state machine. It consists of a sets of states and moves within these states dependent on input data. But what if its not the states which are going to be changes dependent on input, but that it is a lot of data that is going through a chain of operations from a source, through various stages of operation through to the end output. The stage machine is more of a paradigm of data flow instead of program comtrol flow. Sine there is so much data to be moved and managed, it may easilly become too big job to be done in one operation, and most of all; too complex to program and manage. The simplest stage machine design is a serie of stages (1-2-3) that processes the data separately, and delivers the output to the next stage. Each stage operating as a black box, not knowing how or chat the other stages do or how they operate, only knowing what input data it contains, and what output data it should produce. With this limitation each stage can be designed and programmed and tested separately, and even replaced with a totally new stage as long as the new one produces the same output. Programming this is a little more complex. Especially when we manage a set of data so large we cannot run everything at once and sending data through pipes or sockets. Storing the data in a database is a simple solution (there are probably many more variants, but this one has several other good side effects). When storing interstage data in database tables, we can easily make web interfaced to display the same data. And that is a big help when debugging such a program. As you see, the data still "flows" through the stages the same way, bus is stored temporarily in tables between the stages. Another good aspect of this approach is the possibility to let one (or many) of the page to be a web page. Like if you want the data set to be manually checked at one point in the process, just make a "checking" black box stage in the data flow, and if you don't want to do the checking, you can let a simple stage program do the work for you when you develop and test the system. Since all the stages are considered black box systems, you have the freedom to program each stage (module, state-machine, engine, package, program, page ...) the way you want, as long as the interchange data is defined good enough.

Eclipse over Santa Clara

2006-07-18T22:39:00.000+02:00

Eclipse has grown the past few years from being a fairly good java development tool to being a fully functional proffessional development suite for many different environment. I have followed Eclipse the last 4 years, and from when Eclipse 2.1 came out in 2003, I was a devout user... Officially 28th march, but I had used it for 2 months already by then. Before 2003 I had had little experience with Java development, but got a semester job as student assistant in a programming course where we used Eclipse as development platform with Java. And that summer, I got a job with making new tests and exercises for the same course for the next year, and finding suitable tools for development tools, design tools etcetera. My choice was of course: Eclipse as development tool, Poseidon for UML as design tool, and Java 2 SE 1.4 as programming platform. Poseidon was ditched later for Dia because of size, speed and the problematic sides of using a commercial product (although for free as with the community edition) in free education. But that was small in comparison with the introduction of Eclipse into the university education. Eclipse has graually been used more deeply, and thoroughly in more courses.

And my use of Eclipse had not yet stopped. In 2003 I began my master's degree in Computer Science, where I studied the consistency problems of Distributed Hash Tables (DHT), and ways of solving it. Once again: Programming platform and development environment; Java 2 SE 5.0 and Eclipse, and Eclipse lived through 2.1, 3.0 and 3.1 before I got to deliver my thesis. I also got quite some experience with LaTeX and Kile there too. Later I got work at Market Monitor, where most of the developers were using Borland JBuilder for developing Java. But as we used an older free version with limited functionality, and Eclipse already was used for PHP Development, I quickly got tired of repeated program switches, and constant swapping on my computer from two that large programs running (and with my own development work needing quite some load too), I swapped to Eclipse 100%. But as the SVN tree was build for JBuilder, and even some of the code was specifically coded for Windows, I have not been able to move to Linux too. But Eclipse is in use, and with the release of Callisto last month, I have finally been able to move even more over to Eclipse. There are also a list of plugins we (and I) use for helping me in Eclipse, and some of those are vital to my work.

Main Development Tools
PHPEclipse: PHP-Eclipse consists of tools for developing PHP scripts and smarty templates with syntax highlighting, integrated web viewing, database and http server control. It's not that big, but with the PHP and TPL syntax highlighting, and autocompletion, developing advanced web pages have become a game.
Eclipse Web Tools: With Eclipse Web Tools, which is not a signle project, but a collection of web and xml-centric development tools, developing pure HTML, XML, CSS, JavaScript etc. files have got their tool. For not that long time ago, we had to aquire MyEclipse to get a lot of this functionality. Not that expensive for the student version, but not the best solution anyway.
Helper Tools
AnyEdit: A small tool for normalizing files for sharing. It can change the file content in non-changing ways like replacing tabs with spaces, removing trailing whitespace etc. It gives less differences in diff view when comparing SVN Head and Working Base file versions.
Subclipse: A Subversion plugin for Eclipse. Enables SVN respository synchronizing and differenciating on the same level as the builtin CVS feature.
External Tools
MySQL Query Browser: The MySQL Query Browser is a simple program to work with a database and debug queries, browse tables etcetera. Can be used for simple modification queries too. But for complex operation, and interactive modification, it is useless.
Tortoise SVN: A simple yet powerfull integration between the Windows Explorer API and the SVN framework. Simple to use, and helps a lot when I don't want to use the Eclipse to manage versioning. Has a very powerfull difference editor and conflict management system.

Its not the meaning to ditch Sun, as Sun's NetBeans has come quite in the shaddow here. NetBeans 5.0 has become a very good competitor to Eclipse, and is an enormous improvement since 4.0. But it seemed that Sun needed the real competitor to make the really good IDE they almost already had. But I have very little experience with NetBeans, and has almost never used it since 3.5 (which I didn't like at all).

Working with Mozilla

2006-07-01T22:56:00.000+02:00

Mozilla Firefox is a very powerfull tool when working with web pages. One of the tasks I have at Market Monitor is to develop features for web-pages that we host for our customers, and to make web pages for internal use only, either at Market Monitor, or for our customers.

But one of the problems I have is to make pages that can be used in all operating systems, at least in all mainstream browsers. This includes Mozilla Firefox and Internet Explorer. So why have I chosen Firefox as my main tool? I have several reasons for my choice, including but not limited to:

Browser Tabbing keeps my desktop clean
Pop-up blocking keeps my desktop even cleaner, and with carefull management I can control what and from where I want anything up.
Theming the browser for small screen sizes makes the browser controls as small as it gets without loosing functionality, and that includes the overall look of it.
The vast amount of extensions that add functionality and modifies the browser in many different ways.

My collection of used extensions is starting to be quite large by now. I have many security and ad-blocking extensions, as I have a lot of extensions helping me in my development (especially when it comes to web-pages).

I'll make a list and review of the most important extensions, and explain how they help me, or why I like to use it.

Usability Extensions
Bookmarks Synchronizer:
Adds the posibility to synchronize your bookmarks to an FTP or HTML respository. Is both secure and fast, but requires that you have a server available to synchronize with.
Mouse Gestures:
Using mouse gestures for common browsing actions makes it possible to browse efficiently without using the keyboard at all. Fast and easy to learn.
Editus Externus:
Edit the content of input boxes in mozilla with an external editor of your choice. Practical for writing blogs and templates for posting on the web.
SearchBar AutoSizer:
Tired of having a large search bar taking place in the main toolbar when not using it, and that when you use it, it's always to small? The autosizer makes it very small per default, and extends its size when writing the search.
Download StatusBar:
Keep track of current downloads while surfing. The small statusbar is placed by default right above the main statusbar, and contains the main functionality of the download sidebar or window.
Extended Statusbar:
Adds more information to the default statusbar, like number of images downloaded, download speed and time, and percent of downloads complete.
Linkification:
Converts plain text links to anchor tags. So I don't have to copy link, open tab and paste it there to open them. Some pages and especially where mailing lists or forum archives are shown have lots of plain text links.
Text-BgColor Fixer:
A combination of having a dark-background theme to my desktop and a small "feature" in combination; input-boxes where backgound color is not set, but font color is. Fizes so the text is readable anyway.
Security and Advertisement
Clear Private Data:
Clears data from cache, saved form data, cookies etc in a simple add-on to the right click menu.
FlashBlock:
Replaces the content of all flash animations with a somple play-button. Keeps most of the worst advertisements from showing up. It's simple to watch those flashes I want to see, as it's just to click on the play icon.
JavaScript Options:
Adds more options to the javascript option panel in the preferences.
NoScript:
Blocks javascript from running, and with an easy to use interface for controlling the enabling and disabling of the downloaded scripts. Adds more or less complete security to the default Mozilla browser.
Web Services
GMail Notifier:
Adds a small icon on the statusbar which shows the number of unread e-mails that resides on your GMail or Yahoo! Mail account. A single click is also needed to log in to the inbox-page itself.
WebmailCompose:
Adds possibility to go directly to your GMail or Yahoo! Mail mail compose page when clocking on a mailto: link. Adds to usability of Mozilla as a mail program too.
BlogThis:
Adds a "Blog This" option to the right click menu which takes you directly to your create post page on your blog.
BBCode:
Adds BBCode formatting to menu when composing posts to forums and wikis and likewise places where BBCode is used.
Development
HTML Validator:
A small development application of it's own. Checks and displays the various warnings and errors that comes on a page. Displays a small icon in the statusbar with a short summary, and opens a window that shows the specific warnings, with proposals for fixing, and can clean up the HTML code for better understanding and reading it. Really the most important tool for a Web Developer.
Web Developer:
Adds a new toolbar with a lot of extra options and functions that can help with developing a good webside. Functionality includes to highlight table borders, formatting etc.
IE Tab:
Opens an Internet Explorer tab inside Firefox. Now I don't need to open IE in it's own windows anymore. Only drawback is that I have to use the IE context menu to control the page, as many firefox controls don't work for the IE Tab.
ViewSourceWith:
Lets you choose an external editor where you want to view the page source with. Especially good if you want to open the page source in your favorite HTML editor for looking at its structure and solutions for making the page (and finding bugs).

There are more extensions, but those are usially not very important. I have extensions for tabbing, more for bookmarks, talkback for htlping the developers of Firefix, and opening downloaded files. In total Mozilla Firefix has a lot of functionality that I can use for enhancing my development work.

Update on Duplicate

2006-06-28T07:31:00.000+02:00

Making your database stay alive and available when you have extreme large amounts of inserts and updates isn't easy. But by batching up your updates and inserts into 1 single query, you reduce the amount of data transfers, and number of transfered SQL statements.

INSERT INTO table_name ( unique_keys, extra_values ) 
VALUES ( u1, e1 ), ( u2, e2 ), ... , ( uN, eN )
ON DUPLICATE KEY UPDATE extra_values = VALUES(extra_values)

... will try to insert N rows, but when the unique key is duplicate, it will only update the given values instead. The row count for this statement is the number of tried inserts + the number of updates. So to get the number of inserts and updates you can calculate by:

updates = row_count - N;
inserts = row_count - 2*updates;

This approach can be used to pure updates too, but you can not add a WHERE clause to limit updates.

Based on experience this approach can reduce the network and database load when updating large amount of rows with different special values. I am developing a system where a large amount of data is moved from a database (MySQL) store, checked for various updates, name hits, compared to other parts of the database etc., then pushed back to the database. The number of inserts and updates during a 'low activity' period easilly reaches 10,000 rows+ per table on ~15 tables, and may get 4-5 (10?) times larger in 'high activity' periods, and this is repeated regularily (each 3 minutes) during working hours.

This aproach has kept the database load to a level not noticable to the system admins, though I have no numbers on this. I would like to get real benchmark numbers on this, but I don't have the time to do so myself.