Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Development guide #215

Open
wants to merge 8 commits into
base: develop
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .buildkite/pipeline.yml
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ steps:
branches: "!master !develop !release-*"
command:
- "python3 -m pip install towncrier"
- "python3 -m towncrier.check"
- "python3 -m towncrier.check --compare-with=origin/develop"
Half-Shot marked this conversation as resolved.
Show resolved Hide resolved
plugins:
- docker#v3.0.1:
image: "python:3.6"
Expand Down
1 change: 1 addition & 0 deletions changelog.d/215.doc
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Added docs/development_guide.md to aid new users hacking on the bridge.
121 changes: 121 additions & 0 deletions docs/development_guide.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,121 @@
Development Guide
-----------------

This guide is for those that want to help hack on the Slack bridge, and need some instruction on how
to set it up. This is **not** a guide for setting up an instance for production usage.

For information on how to contribute issues and or pull requests, please read [CONTRIBUTING](../CONTRIBUTING.md).

## Setting up your environment

This section explains how to setup Synapse, Riot and the Slack bridge for local development.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Synapse and Riot should have their own documentation in their own projects. We run the risk of it being outdated very quickly here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We've made a conscious choice to make the guide follow-able without leaving the project. The steps are sufficiently noninvasive that they should work for the near future. If the steps do change, then we would update this guide accordingly.

Part of the reason for the guide is to be a one-stop shop on how to setup a developers environment, rather than spending time learning about each project.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We've made a conscious choice to make the guide follow-able without leaving the project. The steps are sufficiently noninvasive that they should work for the near future. If the steps do change, then we would update this guide accordingly.

Part of the reason for the guide is to be a one-stop shop on how to setup a developers environment, rather than spending time learning about each project.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I seriously disagree with that decision. For one, I'm not reviewing the same blobs of text over and over, and I'm sure not expecting that anyone goes around to update all the install guides for all the projects.

Pointing to an existing guide has a two major benefits as far as I'm concerned:

  • The project retains control over which installation method is preferred. The upstream projects (namely Riot and Synapse) are likely to want to change their preferred installation methods, particularly Riot in the nearish future. By nature of the relationship between the bridge team and the auxiliary projects, the auxiliary projects would need to inform the bridge team that their docs are now outdated, leading to a lot of suddenly outdated documentation.
  • The steps are in a single canonical location. There's several bridges and other projects which would need this same guide: by rewriting the same words over and over, we're duplicating information. Pointing to the project's own guides means we don't have to repeat it.

In short: please let's not make more work for everyone.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My problem with keeping the setup instructions in their own home is that the specific requirements for this purpose (setup to create a dev environment for hacking on the Slack bridge) are not made clear by the instructions for those projects. The way it is right now - with very minimal steps needed for each case, plus a link to the canonical sources is much more effective. See this example: https://github.com/matrix-org/matrix-appservice-slack/pull/215/files#diff-c570356b15a684fa012ea55e4347e51bR73, which I found very readable.

What's currently not readable is having the dev setup guide link to the README and DATASTORES pages, which are much more general pages that go beyond the scope of what's needed for a dev environment setup.

It may be more work to maintain these duplicated blobs (I'm not sure how much more work it would be), but right now it's too complex to get started hacking on this project, so more streamlined docs are neeeded.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Setting up a Synapse server for development or for public is largely the same, barring a few config changes. I seriously don't see a problem with saying "Install Synapse as per [their instructions] then add the appservice's registration file to the config. If you're not sure how to do that, [click here]". Both links would (and should) be to the Synapse repository.


### Prereqs
Half-Shot marked this conversation as resolved.
Show resolved Hide resolved

Ensure at the minimum you have installed:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't we also need python if we're explaining how to set up Synapse? Ignoring the fact that I disagree with us telling people how to set up Synapse of course.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not if we're doing it in a container.


- NodeJS ( [nvm](https://github.com/nvm-sh/nvm) is a good tool for this)
Half-Shot marked this conversation as resolved.
Show resolved Hide resolved
- Docker (optional, but it will make your life easier)

For the sake of making this easier to follow, we will create a directory in the home directory
turt2live marked this conversation as resolved.
Show resolved Hide resolved
called `slack-bridge-env`:

```bash
mkdir ~/slack-bridge-env
cd ~/slack-bridge-env
```

### Setting up Synapse

Largely to setup Synapse, you can follow https://hub.docker.com/r/matrixdotorg/synapse and just ensure your data directory points to `~/slack-bridge-env/synapse`.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a change of voice/tone here which is a little jarring: the previous section uses a voice where the guide is the user ("we", "our", etc) but here it changes to be instructional ("do this", "you", etc). Personally I prefer the more fuzzy feeling of the guide saying "we", "us", "our", and such.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The docker commands in the hub.docker.com docs need to be changed. I recommend including them explicitly here with the needed changes, for example:

docker run -it --rm \
    -v ~/slack-bridge-env/synapse:/data \
    -e SYNAPSE_SERVER_NAME=localhost \
    -e SYNAPSE_REPORT_STATS=no \
    matrixdotorg/synapse:latest generate

docker run -d --name synapse \
    -v ~/slack-bridge-env/synapse:/data \
    -p 8008:8008 \
    matrixdotorg/synapse:latest

You will want to make sure your server name is something that routes to your local box. I tend to use the devbox hostname, but `localhost` is also sufficent.
You should enable registration in the `homeserver.yaml` to create a testing user.
Half-Shot marked this conversation as resolved.
Show resolved Hide resolved
You will need to add an appservice registration file in the future, but it is not imprtant for now.

### Setting up Riot

Setting up Riot is also quite straightforward:

Create a new config called `~/slack-bridge-env/riot-config.json` with the contents needed to set defaults to your local homeserver.

As an example:

```json
{
"default_server_config": {
"m.homeserver": {
"base_url": "http://localhost:8008",
"server_name": "localhost"
}
},
"disable_custom_urls": false,
"disable_guests": false,
"disable_login_language_selector": false,
"disable_3pid_login": false,
"brand": "Riot",
"defaultCountryCode": "GB",
"showLabsSettings": false,
"default_federate": true,
"default_theme": "light"
}
```

Finally, run `docker run -v /home/will/git/scalar-env/riot-config.json:/app/config.json -p 8080:80 vectorim/riot-web` to start your Riot instance. You should be able to register a new user on your local synapse instance through Riot.

### Setting up the bridge postgres
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
### Setting up the bridge postgres
### Setting up the bridge with PostgreSQL

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also might be relevant to link to digital ocean's managed databases thing. It's a fairly inexpensive option for people doing lots of development

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's probably out of scope for a fairly restricted guide to get yourself writing simple PRs. The intention is for users to expand their knowledge by themselves after they have an environment set up, so perhaps it can be part of further reading.


Setting up the postgres instance for the bridge is also quite easy.
Half-Shot marked this conversation as resolved.
Show resolved Hide resolved

```bash
docker run -d --name slackpg -p 59999:5432 -e POSTGRES_PASSWORD=pass postgres
```

You should also create a new database in preparation for using the bridge

```bash
sudo apt install postgresql-client # Ensure you have psql installed
Half-Shot marked this conversation as resolved.
Show resolved Hide resolved
psql -h localhost -U postgres -W postgres
Half-Shot marked this conversation as resolved.
Show resolved Hide resolved
```

And then on the postgres shell
Half-Shot marked this conversation as resolved.
Show resolved Hide resolved

```sql
CREATE DATABASE slack
```

### Setting up the bridge

Clone the bridge repo to a place of your choice, again for simplicity we will
use the env directory. Make sure you clone your **fork** and not the upstream repo.

```bash
git clone [email protected]:your-github-username/matrix-appservice-slack.git
cd matrix-appservice-slack
git remote add matrix-org [email protected]:matrix-org/matrix-appservice-slack.git
git checkout matrix-org/develop # Always base your changes off matrix-org/develop
npm i
npm run build
```

The steps above should make sure you are running the latest development version
and have built the typescript. To ensure that it's all working, you can run
`npm test` which will run the unit and integration tests.

You can follow the instructions in the [README](../README.md) to generate and update the registration file, as well as creating a testing Slack app and workspace. You should enable RTM support in the config, as Slack will not be able to push events to your local bridge.
Half-Shot marked this conversation as resolved.
Show resolved Hide resolved

Make sure that you copy the generated registration file to `~/slack-bridge-env/synapse` and add an entry for it in the `homeserver.yaml` before starting the synapse container.

## Making changes

Whenever you want to make changes to the codebase, you must:

```bash
git fetch matrix-org
git checkout matrix-org/develop
git checkout -b yourfeaturname
Half-Shot marked this conversation as resolved.
Show resolved Hide resolved
npm i
```

You should always work within the `src` directory. It is helpful to have a code editor setup with linting enabled so you can see mistakes as you work. Always remember to run `npm run build` before commiting or testing so that you know the latest changes work.
Half-Shot marked this conversation as resolved.
Show resolved Hide resolved

Before commiting your work, ensure that the tests pass locally with `npm test`. If you are making changes to packages, ensure that `package-lock.json` is included in the commit.