Lipstick on a Pig

I've just read the following article claiming to make the case for why "public services" need to be run entirely by the state without privatisation. Link It is full of very serious errors which I will examine, not least of which is the concept of "public service".

First off, let's establish a few things:

1) People Suck (TM). It can be trivially observed that some people are bad and some are good. If this were not true then there would be no crime, no need for a Police force, etc because everyone would agree on a moral code and we'd All Get Along (TM). Some people are selfish, dishonest, greedy, misogynistic, racist and so on down a list of negative traits. This is not to say that there are not many people who are great (there are loads, some of which I am deeply lucky to count as my friends) but People Suck is quicker to type than Some People Suck Most Of The Time But Most People Are Decent Until The Chips Are Really Down, Then Most Folk Will Look Out For Themselves And Those Important To Them. This is simply an empirical observation of human behaviour.

2) Capitalism is the exchange of goods and services for mutual benefit This exchange for mutual benefit is what separates humanity from universally being subsistence farmers, growing and making every single thing they need themselves. See here. Capitalism is not based on greed, neither is it evil. If fact it is a positive-sum game. Greed and exploitation do not arise from capitalism, they are simply explained by tenet 1: People Suck. If you want a well written explanation of how complex economic activity arises spontaneously without any requirement of planning by a state or any other agency, try I, Pencil

3) Capitalism generates wealth, governments consume it. Governments do not have any money of their own, they do not invest and get paid dividends, have savings accounts or anything else like that. Governments confiscate money from people by force, money that other people have earned through their efforts. This is politely known as taxation. This is not a comment on whether taxes can be moral or are justifiable but an empirical observation. If you do not pay the taxes the government legally requires you to, you will be arrested, prosecuted and imprisoned. The only other source of money for a government is borrowing but since this must eventually be repaid using tax revenues it is clear that a government only ever obtains money through force.

4) Only people act. Companies, governments, the public sector are made up of people. Anything they do, good or ill, is not done by the nebulous entity of "company", "government" or "public sector" but by people choosing to do it. It is essential in acknowledging this to remember tenet 1: People Suck.

People may not like these statements but I invite anyone to demonstrate that they are factually inaccurate.

Let's get on with the content of the article:

The primary aim of a public service is to provide a service to the public. This service exists to avoid negative social impacts and protect crucial social utilities from the instabilities of capitalism.

Within living memory it was considered basic common sense that essentials like food, water, energy, access to health services, housing, sanitation and sewage, social care and core manufacturing industries were too important to expose to the volatilities of the free market.

"Within living memory it was considered common sense". By whom? Within living memory (and still today) gay and lesbian people were persecuted by some parts of society. It would more honestly be written "I believe". This is most charitably viewed a case of the logical fallacy argumentum ad populam.

Aside from this practical view, there were also two core value statements:

1) A person or entity should not seek to profit from a person’s need to eat, heat their homes, drink water, be treated when sick or have a roof over their head.

2) A person’s access to such necessities should not be based on their ability to pay.

Value statement 1 is so staggeringly naive of how the real world actually works I am astounded. It doesn't stand up to any reasonable scrutiny. If no person or entity can profit (i.e. benefit from doing work) from providing the basic human needs of food, water, heat and shelter then the only way to provide them is through altruism. The idea that a society will provide all of these freely to everyone no matter how much they eat, drink or leave the doors and windows open is immediately, completely destroyed by tenet 1: People Suck. The pro-state argument states that these can be provided by the state instead.

It is also destroyed by the fact that every single iota of economic activity in a society is based on expenditure of energy. Food is grown, water is purified, heat generated and shelter constructed by expending energy. When a farmer produces a crop various inputs are required aside from seeds, rainwater, sunshine: fertilisers, possibly pesticides, farm machinery to plant and harvest, fuel to run them, transport to take the produce to be processed into packaged products for the consumer. All of those things require energy in their manufacture and operation. Since no person or entity can be allowed to profit from providing food, it follows that they will have nothing to trade for the goods and services, and therefore the energy, they need to produce the food (or purify and deliver the water, provide shelter etc). Energy cannot be produced without cost so no private sector entity will produce it (since no benefit can be derived from doing so) and it would theoretically fall to the state to generate all the energy.

A counter argument would be that the state only generates energy for the provision of food, water, heat and shelter. This is simply impossible as the processes that ultimately create these things are inextricably tangled with processes that create other things. Production of e.g. tyres for agricultural equipment or transport to consumers are made at the same factory that makes tyres for cars, motorcycles etc. The same argument applies to manufacture of chemicals whether used directly like fertiliser or indirectly like fuel, steel, plastics, glass, everything really. Any attempt to gather sufficient information to segregate costs for human needs from costs for non-essentials would inevitably fail due to the nearly infinite complexity of such a mutually interdependent system. Thus the state's only option is to provide the energy for free.

Value statement 2 is a restatement of value statement 1. If no entity can profit from provision of the needs then no payment can be taken and ability to pay is irrelevant. What about providing at cost? These goods cannot have a cost since no-one in the production chain can profit. The only cost is therefore borne by the state which produces all the energy required for all economic activity. However, in order to acquire the resources to generate the energy the state must raise money through taxation of economic activity. Economic activity that provides human needs is inseparable from economic activity that provides non-essential so either all economic activity is taxed (taxing human needs) or none is. Since taxing human needs violates both value statements then no economic activity can be taxed and therefore no tax revenue can be raised, no energy generated and thus no economic activity at all. Thus the entire system necessary to support the value statements is negated by reduction to absurdity.

This has ended up rather longer than I envisaged so I'll split it into more than one post.

Almost useful

A milestone has been reached! A (single frame, non-JPEG) image now displays in the UI. OK, so it only really works for images whose dynamic range doesn't overflow the pixel intensity and cause sharp white-black transitions and there's no interpolation of the image to fit the display or scaling of pixel intensities to fit the width and centre but it's on the screen damnit :P

Intensity scaling and multiframe I can see how to do. Interpolation is simple in theory but the bugbear is going to be JPEG. I'm pushing that squarely onto the back burner for now as the only data I've seen with JPEG encoding is the Osirix example stuff. For now I'm pleased that I can build Hastur/DicomH on my work computer and actually have something that's useful. Admittedly it's more useful for looking at the raw data than the images right now but I've not seen any other program that allows me to look at the raw so easily, free or otherwise.

Proof is what we like so it's time to pony up the screenie... You can see the white-black wrapping issue clearly in the CSF and the skin.

Big steps forward since last time:

• Study (patient) list now reacts to item selection by populating the series list
• Series list reacts to item selection by putting the images for the series into the right hand panel
• The slider below the right hand panel allows the user to move through the images in the series

Two immediate problems I need to fix. The current SOP instance is rendered to text on the "Raw" tab no matter if the tab is visible or not. Since this is primarily concatenating immutable strings to produce new immutable strings, this is rather expensive in terms of repeated memory allocation and garbage collection. This is fixable as soon as I can figure out which events to intercept on which windows. The second one is rather nastier. Trying to render certain images causes the stack to grow beyond 8MB. I suspect a space leak from incorrect processing of the pixel list. Fixing this may involve a fold, in my imperative mind one of the darker and more mysterious parts of Haskell along with monads.

There's so much yet to do. How quickly it gets done depends on how good I am at resisting the draw of Lord of the Rings Online :P

Appearances Deceptive

Another post, another screen shot. Almost looks like a real app :P

Differences are actually quite large though and quite a bit under the UI. An Sqlite3 database holds minimal patient, study and series info. Directories can be parsed for all their contents, recursively descending into sub-directories if desired. It's passably fast at parsing files and importing into the database too, at a bit over 1GB/min. That seems to be IO limited though as the CPU usage on my quad core peaks at about 10% much of which is system time. That will slow down as more data is inserted per file into the database so we'll have to see how bad it gets. Quietly pleased that it went through a lot of data without barfing once :)

The patient list does nothing but scroll and look pretty so far. The decoded file in the bottom right panel is one that was read independently of the database and patient list. There's no way to navigate to a file from the patient. Yet. Not least because individual files/images are not stored in the database yet...

I downloaded a whole bunch of the example files for Osirix only to discover they're mostly using the JPEG2000 transfer syntax. Arg. Vaguely fortunately, only the pixel data uses the actual JPEG2000 compression and the rest of the data is explicit VR little endian. Considerable staring at the standard and a hex editor, the addition of a new element decoding function (about 10 lines) and Hastur now reads all the metadata.

The pixel data itself is going to be a lot more trouble. It's actually held as an encapsulated document in the pixel data tag. The encapsulated document is a series of fragments encoded in a similar way to an SQ element. You can see the hex that delineates the individual fragments in the screen shot, the delimiter is "feff00e0". It won't be hard to write a parser to chew through the encapsulated document but since I don't have a JPEG2000 codec, that will wait.

Next up is storing files and images, linking bits of the UI together and figuring out the widget layout black art.

Tough Test Postponed...

...since it's far more important to have a custom icon on the main window of your UI :)

Much reading, an install of wxWidgets and wxHaskell, much more reading and many coffee later and Ihave something that might yet grow into something useful.

Since I both suck at naming and am quite fond of Cthulhu mythos, Hastur now exists. Terribly amusing pun on Hastur as he's the "One That Must Not Be Named". Hohoho. :P

Screenie here showing the first part of the aforementioned 40MB DICOM file.

BlackMage icon from Final Fantasy.

Shambling Onward

The DICOM library has ballooned somewhat since last time :/ On the upside, the basic parser can now handle both of the simplest little-endian encoding schemes, all the 30+ possible value representations (VR - tag content type) and even nested sequences (both explicit and undefined lengths). I need to think about the exposed API and how to make it usable and also about how to get rid of the inelegant staircases of if-then-else in some methods. Ugly but it does actually work. This both surprised and pleased me the first time it ploughed, without error, through a 40MB multiframe file chock full of nested sequences and dumped it in readable form onto the console.

The library is now split out into modules with defined purposes (bytestream parsing, data dictionary, pretty printing and file reading) and only a very small amount of code has to live in IO. The rest of it is pure, albeit in fairly imperative form.

The implicit little-endian encoding is going to cause me a great deal more typing. Unlike the explicit LE encoding which supplies the VR in the tag itself, the implicit encoding expects you to just know what VR corresponds to each numerical tag. The only solution I know to that is to build a dictionary with the numerical tag as the key. This works as expected with the side-benefit that I can store the tag's textual description along with the VR for use when pretty printing. I do wonder how time consuming this will be to construct every time the library's used. There are a huge number of tags defined in DICOM of which I've used 300 and they took long enough to enter. Perhaps that is the price for trying to decode such a huge standard :/

A tough test to come is whether all this data can be successfully re-encoded into a bytestream then a file and read by an existing DICOM library. That will have to wait though, I need to see what I can do with the in-memory representation and that means it's time to build a UI app to look at pretty pictures in. wxHaskell here we come...

Baby steps in Haskell

After lots of reading and sometimes comprehending over an embarrassingly long time, I now have a very limited but actually working piece of code in a functional language.

Usually, when trying to learn a new language, the hardest thing is deciding on a vaguely pointful thing to build as the learning experience. Not this time. I wanted to create a library for dealing with DICOM medical imaging data. The format's complex enough (lots of optional stuff, various encoding schemes, heirarchical object layout, etc) to be interesting and there are few really good parsers out there for it in procedural languages I'm used to.

There probably still won't be a good one for Haskell when I've finished but I'm hoping to have learned a lot. It's certainly been very hard going getting even this far but the following is a fragment of the output from my first efforts:


Prelude: 0000000000000000000000000 ... 000
Magic: DICM
Metadata:
[ (0002,0000) UL [4] "\192\NUL\NUL\NUL"
, (0002,0001) OB [2] "\NUL\SOH"
, (0002,0002) UI [26] "1.2.840.10008.5.1.4.1.1.4\NUL"
, (0002,0003) UI [56] "1.3.12.2.1107.5.99.2.1951.30000009050612303648400002557\NUL"
, (0002,0010) UI [20] "1.2.840.10008.1.2.1\NUL"
, (0002,0012) UI [20] "1.3.12.2.1107.5.99.2"
, (0002,0013) SH [16] "XXXXXXX"
]
DICOM:
[ (0008,0005) CS [10] "ISO_IR 100"
, (0008,0008) CS [22] "ORIGINAL\\PRIMARY\\M\\ND "
, (0008,0012) DA [8] "2009XXXX"
, (0008,0013) TM [14] "140340.156000 "
, (0008,0016) UI [26] "1.2.840.10008.5.1.4.1.1.4\NUL"
.
.
.
]


Doesn't look like much but it's decoded the group and element info (0002,0001) etc, the value representation (UI, CS, OB, etc) and correctly parsed the right number of bytes for the value. It's also split the file into the encapsulation (prelude, magic and metadata) and the actual DICOM object.

Ok, there's no error handling, it only knows one encoding scheme, doesn't grok heirarchical objects and there's no error handling but it's only 140 lines of code. A good third of that formats it nicely for printing on the console though so under 100 lines for a parser just blows me away. For what it does, that's probably excessively verbose but first it needs to work, then it can be refined and finally, if necessary, it can be made faster.

I don't think I've ever been so pleased with so little code before :o Haskell is very expressive but OMG has it been hard work learning to write even this little bit :/

Truth Stinks

I wish I could claim this as my own:

"You can't polish a turd but glitter sticks to one extremely well"

I'm sure everyone can think of at least one unpolished turd in their lives, whether nationwide political or "that pillock at the office", which the glitter has stuck to but somehow doesn't disguise the awful smell...

Personally speaking, the quote reminded me of the last 13 years of Labour government and deluded worship of an invisible sky fairy, er, I mean, religion.