Monday, March 02, 2015

Net Neutrality (Yeah!), a new distributed wiki, and my current Clojure project

It is, IMHO, a very good thing that net neutrality now has the force of law behind it. The Internet is the most important artifact for sharing information ever and is worth protecting as a neutral platform. Even though there is a lot of value in walled gardens created by Google, Facebook, and Apple I still hope to see many more systems developed that support individual control of our own data and better support for privacy.

I think that Ward Cunningham's new Federated Wiki (on github: is a very interesting idea for combining local storage with federated sharing of content. Something to keep our eyes on!

I have been working on a combined document repository and general research tool that is tailored to my own needs but I will also sell it as a low cost commercial product (or I might make a compiled version free with the source code available with a commercial license for inclusion in other products - I am not sure yet). I want a system where I can store PDF (and perhaps other formats) files from eBooks I have purchased, papers published on the web, etc. with the usual search and annotation functionality. I am writing browser plugins for Firefox and Chrome to let me clip material from the web (stored with source URL and text from multiple selections) as JSON data for ingestion into my system. The final layer of functionality is support for research notebooks that can be used to collect references to local and web sources, along with my own notes and writing. My system is mostly written in Clojure and ClojureScript (with some Java and JavaScript). I still need to design and implement sharing by exporting research notebooks as PDF files and an easy to reuse JSON dump format, and a mechanism for making parts of a personal knowledge repository public via a read-only web interface.

The first version of my product will be for individual use (simple install on a personal server, or run locally on a laptop) but I am interested in evolving it into a federated system for use by small teams. A federated system is also useful for the single user use case: for example, sync content from a laptop to a server.


Dmitri Sotnikov said...

I had a similar need to generate PDF documents from structured data a little while back, so I ended up writing a library for it and wrapping it up as a service that accepts JSON and returns a PDF document seen here might come in handy. :) There's also pdfmake for a client-side solution.

Mark Watson, author and consultant said...

Thanks Dmitri, I will check out your library.