Q2 - Progress Report

What did you do to reach this Milestone?

The Liquid investigations project has two tech approaches:

The “Cloud Approach”, where we host the liquid software bundle as a ready-to-use web platform The “Federated Approach”, based on ARM class devices with preset configurations. Our milestones refer to both approaches.

  1. We further researched the best software candidates to be included in the software bundle. We evaluated the viability of the Matrix chat service on ARM and concluded that it is a better candidate than RocketChat. We also evaluated HackMd as an alternative to Etherpad. Additional details of all software evaluations can be found in the architecture documentation here: https://github.com/liquidinvestigations/liquidinvestigations/wiki/6.-Implementation-View

  2. Chat, wiki, and filesharing were integrated into the software bundle. Details of each selected application are available in the architecture documentation here: https://github.com/liquidinvestigations/liquidinvestigations/wiki/6.-Implementation-View.

  3. We have set up a bare bones admin app (https://github.com/liquidinvestigations/core) serving as a central user database for the software bundle. A complete design of this Admin UI is something we will focus our efforts on in Q3.

  4. User accounts are synced across all applications in the bundle (no manual account creation is necessary for any specific service, all account creation is handled through the liquid-core UI)

  5. We have implemented OAuth support in Davros (our chosen file sharing application), giving us secure file sharing between account holders on a node. Davros using OAuth meets our Q4 requirements for Single Sign On, and will serve as a model for implementing OAuth/SSO support for the other applications in the bundle. Details can be found here: https://github.com/liquidinvestigations/davros; More specific details about the OAuth implementation can be found here: https://github.com/liquidinvestigations/davros/tree/feature/oauth-middleware#oauth.

  6. We’ve implemented docker support in our ansible setup for both ARM and cloud deployment; This speeds up image creation time and gives us better control over the individual application versions and dependencies at image creation time.

Most important activities:

  1. We have set up a working demo for the cloud service at https://liquiddemo.org. This demo includes all of the software applications listed in the Architecture Document.

  2. We are hosting the liquid software bundle on an internal server for our associate organizations to test. We’ve manually set up users with usernames and passwords to access the demo apps.

  3. HTTPS is set up and working for all applications in the cloud bundle.

All code necessary to support the deployment of the cloud service at liquiddemo.org is available in the ‘demo’ branch of liquidinvestigations/setup here: https://github.com/liquidinvestigations/setup/tree/demo

As shown in our presentation video [ https://youtu.be/BDOdf1XsyxY ], users are able to log into our demo server and access all the liquid software apps with their credentials.

Most important activities:

  1. We have written an automated image build system, replacing our manual image building process. We now have a repeatable automated process that will produce images for different architectures (various ARM boards + x86_64). Technologies that were used are qemu, libvirt, kitchen. Details are here: https://github.com/liquidinvestigations/buildbot. Using this automated build system, we have set up nightly image builds for x86_64 and arm64 architectures. These builds can be found here: https://liquidinvestigations.org/images/nightly.

  2. We have implemented a discovery method that allows visibility between nodes that reside on the same network. When more than one liquid node is on the same network, each node’s home page will display a list of other nodes on the network. This implementation of discovery meets all of our requirements for Discovery for both Q2 and Q3, in that is decentralized and does not rely on a central directory server.Discovery is implemented using zeroconf (avahi). Details of the implementation can be found in the ‘discovery’ repository. (https://github.com/liquidinvestigations/discover)

  3. We have devised a configuration file format that supports configuration of ARM-based nodes for all of the user scenarios defined in the Architecture document. See here: https://github.com/liquidinvestigations/liquidinvestigations/tree/master/sample_configs and here: https://github.com/liquidinvestigations/liquidinvestigations/wiki/7.-User-Scenarios.

Can you please describe the stage of your project and the achievements realized so far?

In Q1, we mostly focused on evaluating various existing software services to incorporate into the liquid software bundle and determine software constraints for hardware resources (ARM), connectivity, security and usability.

In Q1, we demoed how 2 ARM boards, running the liquid software bundle (Search and Annotations) can work together in a basic network setup (the first board was configured to start a WIFI access point and the second board was configured to accept VPN connections from other boards).

In Q2, we focused primarily on build infrastructure and application integration, as well as node discovery. Each of the following accomplishments are explained in more detail with links to documentation above:

Bare bones admin UI OAuth server Shared users between applications Initial OAuth support for applications (Davros) HTTPS support for cloud solution Dockerization of applications Automated image build system Node discovery (zeroconf/avahi) Node configuration file format and initial implementation of config system

Additionally we worked on both internal and external communication streams and maintaining clean project documentation following free and open source standards and licensing.

Github landing page:
Task management system:
Github issue repository:
Architecture Documentation:
Project website:

What evidence can you provide to prove that you reached the Milestone?

A. Cloud Approach Evidence for completion:

Video demo of logging into the server and using all the liquid apps (search,annotation, chat, wiki, file-sharing) [ https://youtu.be/BDOdf1XsyxY ]

B. Federated Approach Evidence for completion:

A config script working with an OS image https://liquidinvestigations.org/images/q2-exit/ NOTE: please review the README.md in this directory for usage instructions Discovery method between boards https://github.com/liquidinvestigations/discover

What were your biggest impediments so far?

The considerations below are not impediments per se - but are aspects that we need to factor in, requiring us a certain degree of flexibility and constant monitoring the ARM class market. ARM boards are constantly evolving. Capacity and scalability grows with the hardware. So far we have evaluated and tested various board types. Besides memory, storage type and capacity, we find that a good documentation and community support for the boards are also essential.

While the Khadas Vim can be set up as a Wifi access point (in comparison to the Odroid C2) the documentation and community support for this type of ARM board is minimal. We are now investigating https://www.pine64.org/?page_id=7147 - as another alternative.

Another issue that we highlighted in the Q1 report was the unavailability of certain boards in online stores, making the hardware ordering process more difficult than needed. Our engineers work remotely from different locations (Bucharest, Berlin, Ontario) so shipping ARM boards for testing purposes to various locations brings with it certain delays.

What caused problems?

During the course of Q2, we made a number of time trade offs. Our primary goal has been to deliver a usable product with an eye to future milestones. As such, we gave time priority this quarter to the implementation of a number of project components that meet future milestone goals (see below).

Because of our time investment on these implementations, we were unable to complete the integration of ARM-based chat and file-sharing applications. These integrations are first on our priority list for Q3.

The Q3 github issues for integration of ARM-based chat and file-sharing are numbered #40, #42, #51, and can be reviewed here: https://waffle.io/liquidinvestigations/liquidinvestigations

The components that we gave priority to are as follows:

‘Buildbot’, an image building system which will be used throughout the project to deliver repeatable, easily produced system images for various arm and x86 architectures. (see: https://github.com/liquidinvestigations/buildbot)

‘Discover’, a decentralized, server-less board discovery mechanism which meets our Q3 goals for board discovery. (see: https://github.com/liquidinvestigations/discover)

An OAuth/SSO-ready fork of ‘Davros’, which meets our Q4 single sign-on milestone requirement and will serve as a model for future OAuth/SSO integration work. (see: https://github.com/liquidinvestigations/davros)

Any other things you want to mention here?

We have continued our series of outreach activities:

Our project (ie. here you can find our project slides: https://tinyurl.com/y76r46yc) was presented within the European Investigative Journalism & Data Harvest conference http://www.journalismfund.eu/dataharvest-conferences in 3 various panels:

This generated substantial interest from various groups, especially the news organizations pertaining to the EIC.network (European Investigative Collaborations) and Theblacksea.eu network. EIC.network is interested in testing the liquid the software bundle as early as Q3. This gives us an extra incentive to finalize the prototype within the defined timeframe and start thinking about scalability and support.

In order to begin collaboration with EIC.network, who will promote the usage of our bundle to all of their media partners, we have started tech meetings between the EIC.network group and the Liquid Investigations core team. We have also made plans for face-to-face meetups and presentations of the technology.

We are aware that the adoption of new technology by users is an important step. In order to avoid new technology shock and rejection by users, RCIJ (Romanian Centre for Investigative Journalism) has begun promotion of the project within the EIC journalistic network. We have had initial face-to-face meetings with a hands-on approach, and the technology is currently being applied to ongoing investigative projects. One such warm up was organized already at an EIC meeting in Brussels in July, and another is planned for October.

We are thrilled that the July 6th issue of Nieman Reports, featured our project and published a large article about our initiative, entitled: ”An Investigative Toolkit for the Post-Snowden Era”.