Java and the Photography Market

Coming from a three years experience in the professional photography industry and a computer science background, I can consider myself as a good observer of the ever changing relation between imaging and technology.

Actually I’m collaborating with Eikona Studio where I had the occasion to work on assignment for the major Italian’s magazines, corporations, events and activities. This gave me the hassle (and the opportunity) to deal with complex and various workflows as well as high demanding image storage and deployment necessities.

Nowadays the daily photographer’s work routine involve a lot of computer work. He does not only need to edit his images, but also store them, backup, deploy to clients, upload to websites and social networks, apply meta-tags, cataloging, etc… while it is quite a diffuse habits for young emerging professionals, for the older and affirmed ones there is still a slow but unstoppable evolution process. Being able to provide practical and flexible tools to simplify those activities is a really good opportunity to watch for as a developer. It could be possible to cover both the market of the tech-savvy newcomers and the old one which need a mass migration.

Although I’m talking about a specific area as it is photography, these problematics are common to all the situation where we have to organize a big amount of media files. In the IT world we label the software which try to deal with it as DAM (Digital Asset Management), unfortunately the actual DAM’s scenario is far from being able to provide practical and user friendly tools. The available software often needs to be adapted by specialized developer and have to run on costly infrastructures, while the medium sized professional on the best case rely on a small NAS cabinet that is going to have big trouble with big Java applications. These aspects (maybe not absolute, but quite common to all the actual major DAMs) lead to a market that is only able to satisfy the needs of medium-big corporations, having them no problem on hiring an entire IT department and invest on high performance servers.

Coming to Java, it seems to be one of the preferred platforms to develop a DAM; by the way we have to figure out how much of this preference is due to the “enterprise friendliness” of the JVM and how much is to be researched in the capabilities of the language itself. The first aspect to consider is that Java Developers are a lot, sometimes cheaper and often with an enterprise background. The advent of Java dialects, the rising of android, the opensource effort to produce good java frameworks and the update to Java 8 are all factors which are freeing the language from its previous boundaries, hence the needs to change some old habits of java programming.

The first thing to do is to avoid to develop application with monolithic cores. With pressing due dates and a closed environments a lot of old school enterprise applications tends to quickly become an enormous monster of code which can be understood only by the people who wrote it and does not deal too much with good interconnection with external resources nor at all with code reusability. Sometimes those big gray “AWT-looking” beasts works quite well too, but when you need to customize them you will practically need to rewrite and recompile enormous pieces of that big core. It is clear that this approach leads to an always worst codebase and a huge and costly effort for every small change to do. These are basic Computer Science concepts by the way, I do not think that old Java programmer are not good programmer, maybe they are, but the environment and the use cases they were usual to face are changing very quickly. In the future there will be no more space for application that works well only on strict boundaries and specific controlled situations. Today everything need to be flexible, extendible, interoperable, cloud and mobile ready.

Another aspect that has pushed Java in the DAM’s market, and that I think will continue to push it, is speed. Java is one of the faster, if not the fastest, high level, garbage collected languages out there. Having to deal with big media file means having to do a lot of cpu intensive tasks; just imagine what it could mean to batch resize 10000 images with Lanczos filter using a scripting language… Java is a statically typed rock solid language and can reach optimal performances (when used correctly) with his high potentials in terms of scalability and parallelization. Due to these aspects some of the majors web based DAMs out there like nuxeo or RAZUNA run on Java, by the way Python alternatives like NotreDAM are just right behind the corner and I guess that in the future there will be other languages competing on this market. Erlang, for example, is resurrecting thanks to its incredible concurrency capabilities: the world of computing is moving towards a highly distributed structure and being the best at parallelizing over the network (and not only over multiple CPUs) is a killer feature thinking to the future. Application are also moving on browsers and the concept itself of a browser is quickly changing: if a language can be understood by a browser it is going to have a huge impact on the next years. In fact node.js is rising at the speed of light. Although it now excels on I/O bound tasks due to his asynchronous structure, I’m ready to bet that libraries such parallel.js and new JS projects such as asm.js, will lead in a really short time to have very good performance for CPU bound tasks in JavaScript. Last, but not least, between Java’s competitors as a server side language there is GO. Made in Google it is like a simplified, modernized, concurrency ready C; besides of the syntax the speed is also comparable to C and Java and I’m sure that the big hype around this new kid in town plus the support of “big-daddy” Google will carry it to a glorious future.

It is clear that Java could not just stay aside watching the world changing, and fortunately it is not what it is doing. Developers tends to undervalue it, but one of the biggest problems with Java is that it is not “sexy”. Coding should be a joy and reviewing source files a pleasure, hence the needs of sexy languages. Dealing with this unresolved desire some smart guys like Guido Van Rossum and Yukihiro Matsumoto in the past years have started developing new languages to resemble all their inner perversions about the beauty of the code. Mixing some of the sexier languages of the past with their modern ideas they gave to life creatures such as Python and Ruby. A beautiful code is also way quicker to develop in and easier to maintain, due to those (and much more) benefits, also the Java community has done great things. Being able to rely on a structure such as the JVM is an enormous opportunity for language architects to develop idioms which, compiling to JVM bitecode, give them both great visibility and a solid result. I’m particularly referring to scala and clojure, but also groovy, jruby and jython.

Trying to figure out what are the features that a good future proof DAM should have we could try to understand the daily workflows of a modern photographic studio as a good starting point. Since now we were used to divide the shooting time from the editing time, by the way it is always more common to use live transfers of the images from the camera to a computer, above all during studio, architectural or still-life works. Being able to directly operate with camera drivers is definitely something to research on. While today data transfer speed (particularly in a wireless environment) could be a bottleneck, develop for the future means to assume that this limitation is going to fade away soon. Once obtained the shoots it’s time to discern the good ones from the bad ones, and so decide what are the images to be edited and archived. The passage from the selection and editing stage to the archival one is, to me, an extra layer of complexity that leads both to be more error prone and to a lot of data transfers which could be easily avoided if everything will be done in the same environment. Professionals are usually scared by the idea of editing in place, and prefer to have solid backups to feel safer; by the way this amount of redundancy is a clear waste of memory. In fact today the modern photo editing programs use to be non destructive, this means that every change is recorded as a serialized list of instruction to be applied to the original bitmap, without touching it. This way an edited version of an image could be stored with only a bunch of textual instruction as overhead instead to have two big slightly different images. I like the lightroom approach: for every images it is possible to create virtual copies and various versions (with their own different set of instructions), without edit directly the original one nor copying it multiple times. What i do not like of lightroom (besides being closed source) is that it is quite difficult, if not impossible to use it properly as a shared resource. While there are various software in the photo editing landscape which get their job done quite well, none of them seems to be ready to scale to multiple users and multiple locations.

Once the images are edited and archived, we need to be able to share them, deploy to clients and also being able to find them years later. Thinking as a developer I imagine resized images being sent in output as streams instead of being saved somewhere, this way the application could decide to send that stream directly over the network on the fly or if save it to a local file. We should avoid to use temporary folders anymore and also feel confident about the increase of network speeds. It is important to be able to create personal workflows, such as take those images and send a small version to facebook with a watermark, then send a hi-res version to that printing service and upload CMYK TIFs to the client’s ftp. It is impossible to know in advance how complex could be one of those workflows, hence the necessity to give the user easy tools to edit them, basically it is always about resize, rename, watermark, upload, etc… those tasks should be core features of every photography targeted DAM; but also it is vital to be able to mix them as needed (easy scripting, visual programming, etc…).

When it is time to archive for future using, but also when we want our images to be easy searchable on the web, we need to deal seriously with metadata. It is incredible how difficult is to find photography software with a good structured keywords support (read about them on controlledvocabulary.com). Tagging and describing images is tedious, but needed, so it is a good idea to invest on researching good methods to deal with it. I think about big data analysis and machine learning to suggest keywords or automatically apply them, but also a rock solid way to edit metadata on the fly, from everywhere and with no effort. Tagging should be considered a core feature and we should be able to include it as a workflow step; imagine for example something like take this bunch of images, apply those tags, then prepare a resized version with keywords collected from [gettyimages](www.gettyimages.com) similar pictures and upload them to istockphoto.

One aspect that we should not underestimate is that clients are changing too, they want to see things while they are still happening: often we have to uploads tons of images to online services such as smugmug just to create a quick gallery to let them make a selection. Once they point out via mail the images they want to be edited we edit them, then we need to create a new gallery with the final images ready to be downloaded. This is clearly a bad workflow: in my opinion a good platform should permit you to create web galleries on the fly and just allow external users to apply a small subset of metadata to your images (such as star or color rating), then one could work on those images directly and once saved their new version should be automatically available on the same gallery.

I’ve started to think what could be the best way to implement those kind of services, and I’m also very interested on hearing from you any ideas about it. Basically I think we should treat the whole thing as a strong, solid, minimal backend that implements all the core instructions and offer them to clients trough clear APIs. The server should be wrote with cloud computing in mind and be ready to be distributed over different nodes. Offering a good RESTful interface will permit to interact with the rest of the world in a clear and consistent manner. Noting that can’t be done with JAX-RS or with scalatra. Java also offer awesome tools to treat concurrency and parallelism, we can rely akka a great implementation of the actor model for both pure Java and scala and of course on hadoop with his distributed filesystem HDFS. I also would give a try to Clojure, a lisp dialect that runs on the JVM developed with concurrency in mind.

Regarding the clients it is vital to avoid the error to make a big swing application that try to make it all. Clients are variegates as they are the use cases, so the main objective to aim for is a clean API which permit the community to create a lot of interfaces to tackle the different necessities. Obviously to furnish a good and stable application to interact with the server is advisable, but I’d avoid to try to push it over the others, instead I’d present it as a good starting point to start hacking. REST API are widely accepted and perfectly usable as an architecture independent interface on the web, so we put no boundaries on the client structure and core technologies. We could easily implement a web application using a JavaScript framework, or a desktop application with python. Using technologies like node-webkit and phonegap it becomes trivial to share a big amount of the code between a web version, a mobile app and a desktop program. As Java developer we could also write native android apps and, why not, rely on Google App Engine to write a webapp. Good APIs also mean to help the flourishing of a huge amount of plug-ins to interact with third party software and web services, and in the long terms it could really makes the difference.

In those terms it looks like this could be an open source friendly approach, and in fact it is, but it doesn’t mean that it won’t be possible to monetize. This is also a clear win for a cloud hosting company: while a big company could rely on his own server, the medium-small studio will be happy to pay a reasonable price for this kind of service. Probably there always will be some uncovered necessities and the possibility to tailor a client, or also some aspect of the backend, on a specific target could lead to new business opportunities. Also as an independent developer it would be possible to write a good mobile client and make some money from it. I also think that as a Java developer the gold coin could also be on being able to scale it on different network and hardware structures.

Java has still a lot to say, and I think that CPU intensive application like this could be a nice weapon for a Java developer in the future, the most important thing is to adapt ourselves to an ever changing world. While some of my consideration could be labeled as crazy, I guess they will become reality as network speed will be able to deal with 50Mb RAW files. Progress today is way faster than yesterday and to not be on delay we have to start developing with future in mind.

About Me #

I was Born in 1987 in Rome where I’m actually living. After a childhood spent on dreaming to become a scientist, I’ve firstly approached programming languages at 11 when I’ve casually found QBASIC on an old 486 laptop.

Now I’m still studying Computer Science at “Università La Sapienza” in Rome. I should have already finished it, but I’m proudly a bad student. Everything I learn, I learn it by myself or working with inspiring people; but this do not help with exams. Then I’ve also spent 1 year doing my Erasmus at “Universidad de Las Palmas de Gran Canaria”, this neither helps with exams, but being able to live and work in a multicultural environment has no price in my humble opinion.

I’m collaborating with Eikona Photography and Digital Imaging as a Photographer, IT Specialist and Web Designer

 
9
Kudos
 
9
Kudos