What is Driveshaft?

Driveshaft is a Sinatra application built to turn Google Spreadsheets and Google Documents into JSON with the click of a button.

We use it at The New York Times to published structured data for interactives and widgets on NYTimes.com and configuring internal settings. Using Google Drive files lets multiple editors collaborate in a familiar environment.

Goals

Publish to S3 in one step
Version history, ability to restore older version
Publish any file to multiple destinations / environments
Google Drive authentication should be invisible when possible (within a firewall) but explicit when necessary (let users authenticate with their own accounts)
A website for humans, and an API for automated publishing
Heroku (and other Platform as a Service frameworks) deployable

Supported Formats

Driveshaft can convert documents in multiple data formats into JSON. It ships with support for two, but you can easily add more.

Spreadsheet

The default for Google Spreadsheets.

Sheets (or tabs) becomes a top-level object in the output, keyed by sheet name. Each sheet's rows become an array of objects. Column headers come from the first row of the spreadsheet.

Example: Spreadsheet → JSON

Spreadsheet example

{
  "people": [
    { "name": "Amanda", "age": "26" },
    { "name": "Tessa", "age": "30" }
  ]
}

Hide "sheets" (tabs) or columns from the output by appending :hide to its name, as with the notes field and other sheet.

ArchieML

The default for Google Documents.

Uses the ArchieML format for creating a JSON object from the text of a Google Document.

Example: Document → JSON

key: value
test: document

{
  "key": "value",
  "test": "document"
}

ArchieML is based on key-value pairs, but supports more complex data structures as well.

Getting started in 3 steps

The instructions below will help you get Driveshaft up and running on your computer, and / or ready to deploy to Heroku.

Step 1: System Dependencies

To begin, you'll need to have a few programs installed on your computer.

The instructions below assume you are installing on a Mac using Homebrew. If you plan to install on another operating system, you can help us out by documenting the steps and issuing a pull request to our GitHub repository.

Git to download and update the source code.

brew install git

Node.js, npm and Bower for JavaScript and CSS package management.

brew install node
npm install -g bower

Ruby, if your system doesn't already have it.

brew install ruby

Driveshaft has been tested using Ruby 2.2.x.

Bundler for ruby package management.

gem install bundler

Step 2: Download and Run Driveshaft

Clone the Driveshaft repository using git, and move into its directory. (If you don't have git, you can download driveshaft instead.)
```
git clone git@github.com:newsdev/driveshaft.git
cd driveshaft
```
If you're deploying straight to Heroku, you can skip to step 3.
Install the JavaScript and CSS dependencies.
```
bower install
```
Install the Ruby gem dependencies.
```
bundle install
```
Start the server.
```
puma
```

By default, puma will run Driveshaft on http://localhost:9292.

Step 3: Enable Driveshaft to access other services

Google Drive API so Driveshaft can access your files
Amazon S3 file storage service to publish JSON to the internet and keep multiple versions around

Step 4: Deploy (optional)

Deploy to Heroku with these instructions
Or deploy to whatever server architecture you already use

Tour

You can try out our demo deploy of Driveshaft on Heroku.

Driveshaft uses Adcom, an open source set of styles and jQuery plugins created for admin sites at the New York Times. For more information, check out the documentation and the code.

Index Page

There are two sections:

Convert a file by url. Enter a URL or Drive ID to go to a convert page for that file. Optionally list one or more S3 destinations you would like to publish to.
Curated list of Drive Files. Powered by a Google Spreadsheet you can specify in the settings.

File Page

Edit, Convert, Download. The top of the file page links to the source file, and lets you convert + download it using any of the available formats.
Publish to S3. Each S3 destination configured is listed and can be pubished to. There's a list of previous versions, each of which can be viewed or reverted to.

Thanks

Released at the OpenNews Code Convening at the 2015 Write The Docs conference with the support of the Knight Foundation and The Mozilla Foundation.