News

The latest updates about Superdesk

BACK TO BLOG OVERVIEW

Superdesk development environment setup

Photo by Farzad Nazifi (CC BY 1.0)
Photo by Farzad Nazifi (CC BY 1.0)

If you are anything like us here at Sourcefabric, you've also fallen in love with Superdesk and can't wait to download it or help contribute. This platform contains technology that is meant to be built upon, improved, added to and extended, all in view of a development community committed to flexibility and freedom. If this sounds like you, you’ve probably tried to get started with Superdesk and have a few questions. 

What are the different source code repositories? How do I set up a working development environment? Where can I apply changes to both the server and client, experiment on a running instance and run both user and automated testing against my changes? 

These are all great questions and will be answered in this post. We will explain how most of us here at Sourcefabric are working and should be helpful to both OSX and Linux developers.  Let’s start with some basic descriptions of the current Superdesk landscape.

What is the difference between a standard Superdesk install and a development environment?

A standard Superdesk instance can be deployed simply by downloading (or cloning) the main Superdesk repository, and following the installation instructions in the README.md. While this installation will provide a fully operational instance of Superdesk, it will not provide a functional development environment for developers to modify, test, and submit Pull Requests back to the Superdesk core repositories easily.

What's with all the repositories?

Superdesk source code is hosted in GitHub, the most popular online service for building software. We use GitHub, their infrastructure and services makes possible the development of Superdesk... for free!

The entire Superdesk code base consists of three separate repositories each containing a different subset of code and configuration for the Superdesk platform.  This allows for more flexibility and independence in development, tagging, and releasing of each respective component in the Superdesk platform. In general, the three repositories can be described as follows:

Superdesk repository

This is the master repository and contains dependency configuration that links the core (server and client) repositories, and any instance specific configuration.  With this repository you can deploy a complete running instance of Superdesk, both client and server core packages are provided as software dependencies (npm and pip respectively) upon install.  

Superdesk server core repository

This repository contains code related to the Superdesk server. You should fork this repository if you plan to submit changes to the Superdesk server code base (Python).

Superdesk client core repository

This repository contains code related to the Superdesk client. You should fork this repository if you plan to submit changes to the Superdesk client code base (Javascript, HTML, CSS, etc).

How to setup a development environment using symbolic links

One simple way to setup a working Superdesk instance with updatable client and server core files is to use symbolic links rather than compiled packages.  The following steps describe how to remove the packages from the default Superdesk install and replace them with links to your locally cloned core repositories.  In the remainder of this post the following conventions will be used:

$SERVERDIR refers to the server subdirectory on your local machine of the main superdesk repository. On my local machine it currently resolves to: /User/marklewis/Sites/superdesk/server

$SERVERVENVDIR refers to the server’s python virtual environment directory on your local machine you created during the initial superdesk install. On my machine it currently resolves to: /User/marklewis/Sites/superdesk/server/env

$SERVERCOREDIR refers to the root directory of the cloned superdesk-core repository on your local machine.  On my machine it currently resolves to: /User/marklewis/Sites/superdesk-core

$CLIENTDIR refers to the client subdirectory on your local machine of main superdesk repository.  On my machine it currently resolves to: /User/marklewis/Sites/superdesk/client

$CLIENTCOREDIR refers to the root directory of the cloned superdesk-client-core repository on your local machine.  On my machine it currently resolves to:  /User/marklewis/Sites/superdesk-client-core

Step by Step

1. Fork the main Superdesk repository and clone it to your local machine  

2. Follow the installation instructions from the main repository README.md file

3. Fork and clone the superdesk-core repository

4. Remove the existing server core packages created during the initial install

rm -rf $SERVERVENVDIR/lib/python3.4/site-packages/apps

rm -rf $SERVERVENVDIR/lib/python3.4/site-packages/superdesk

rm -rf $SERVERVENVDIR/lib/python3.4/site-packages/tests

5. Make a new src dir in your current virtualenv and link to your clone of superdesk-core

mkdir $SERVERVENVDIR/src

ln -s $SERVERCOREDIR $SERVERDIR/env/src/superdesk-core

6. Fork and clone the superdesk-client-core repository

7. Create global link to npm superdesk-core package

cd $CLIENTCOREDIR

npm link

8. Link your clone of superdesk-client-core to the clients node_modules

cd $CLIENTDIR

npm link superdesk-core

How to make client changes, step by step

First, make sure that you have built everything in the superdesk-client-core directory, not just the main repository client dir.  You only need to do this once (or if package.json has been updated)

cd $CLIENTCOREDIR

npm install

Next you’ll need to setup a test server for the protractor e2e test to run against.  The server is provided in the superdesk-client-core repository.  You will only need to do this once.

cd $CLIENTCOREDIR/test-server

virtualenv -p python3 env (or however you wish to create your virtualenv)

source env/bin/activate

pip install -r requirements.txt

1. Make desired changes to superdesk-client-core cloned files

2. Test your changes manually

3. Run karma tests and jshint

cd $CLIENTCOREDIR

grunt ci

4. Run Protractor (e2e) tests

a. Start server for e2e tests (if you haven’t already done so)

cd $CLIENTCOREDIR/test-server

source env/bin/activate (or whatever your virtualenv is setup as)

honcho start

b. In a new terminal start the full e2e test suite - be prepared, this will take over one hour

cd $CLIENTCOREDIR

./node_modules/protractor/bin/protractor protractor.conf.js --baseUrl 'https://superdesk.sourcefabric.org:9000'

5. If all test are successful, submit a pull request to superdesk-client-core

6. Request code review

7. After code review request pull request to be merged to the superdesk-client-core repository

8. Update Superdesk main repositories superdesk-client-core dependency link, found in $CLIENTDIR/package.json

How to make server changes, step by step

First make sure that you have built everything in the superdesk-core directory, not just the main repository server dir.  You only need to this once (or if requirements.txt has been updated)

cd $SERVERCOREDIR

virtualenv -p python3 env (or however you wish to create your virtualenv)

source env/bin/activate

pip install -r requirements.txt

pip install

1. Make desired changes to superdesk-core cloned files

2. Test your changes manually

3. Run nosetests

cd $SERVERCOREDIR

nosetests

4. Run behave tests

cd $SERVERCOREDIR

python start_behave.py

5. Run python linter

6. If all test are successful submit pull request to superdesk-core

7. Request code review

8. After code review request pull request to be merged to the superdesk-core repository

9. Update Superdesk main repositories superdesk-core dependency link, found in $SERVERDIR/requirements.txt


I hope this helps you to get started with Superdesk, if you have any further questions please join us in the forums, we will be happy to assist you!
BACK TO TOP

Latest Articles

Sign up for free to our monthly newsletter to receive Software updates and news about Superdesk

Before you go

See Superdesk in action with a no-obligation demo for your organisation.

Schedule a demo