Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Install with cargo #29

Closed
wants to merge 7 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 0 additions & 17 deletions .github/workflows/actions/setup/action.yaml

This file was deleted.

10 changes: 7 additions & 3 deletions .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,9 @@ jobs:
with:
submodules: true
- name: Setup
uses: ./.github/workflows/actions/setup
uses: dtolnay/rust-toolchain@stable
env:
LLVM_SYS_150_PREFIX: /home/runner/.wasker/clang+llvm-15.0.0-x86_64-linux-gnu-rhel-8.4
- name: Run clippy
run: cargo clippy --all-targets --all-features -- -D warnings
test:
Expand All @@ -30,6 +32,8 @@ jobs:
with:
submodules: true
- name: Setup
uses: ./.github/workflows/actions/setup
- name: Run test
uses: dtolnay/rust-toolchain@stable
env:
LLVM_SYS_150_PREFIX: /home/runner/.wasker/clang+llvm-15.0.0-x86_64-linux-gnu-rhel-8.4
- name: Run test
run: cargo test --all -- --nocapture
2 changes: 1 addition & 1 deletion Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "wasker"
version = "0.1.0"
version = "0.1.1"
edition = "2021"
description = "Wasm compiler for running Wasm on your favorite kernel"
license = "MIT"
Expand Down
107 changes: 42 additions & 65 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,94 +18,48 @@ This unresolved feature allows Wasker's output ELF file to be **linked with WASI

Wasker empowers your favorite OS to serve as a Wasm runtime!


# How to run Wasker
Wasker compiler is based on LLVM (LLVM 15 currently).

![demo](./doc/assets/wasker.gif)


## Option1 : Use Docker
Clone repository
```
git clone [email protected]:Mewz-project/Wasker.git
cd Wasker
```

Create directory for mount and place input Wasm/WAT file.

Please refer [examples](./examples) for building Wasm from Rust and Go.
Here, as an example, we'll use the already built `helloworld.wat` included in this repository.
```
mkdir -p mount
mv helloworld.wat mount
```

Run Wasker via docker (It may take few minuits to pull image, so you can build image locally by `docker build -t wasker .`)

ELF file will be generated under `mount` directory as `wasm.o`.

```
docker run -it --rm -v $PWD/mount:/work/mount -t ghcr.io/mewz-project/wasker:latest mount/helloworld.wat
[2023-12-28T09:20:27Z INFO wasker::compiler] input: mount/helloworld.wat
[2023-12-28T09:20:27Z INFO wasker::compiler] write to wasm.ll
[2023-12-28T09:20:27Z INFO wasker::compiler] write to wasm.o, it may take a while
[2023-12-28T09:20:28Z INFO wasker::compiler] Compile success
```

## Option2 : Use Devcontainer
You can try Wasker on browser via Devcontainer.
# Installation
Wasker compiler is based on LLVM (LLVM 15 currently).

Start Devcontainer
- Click `Code` -> `Codespaces` -> `New codespace` on this repository page.
- Wait for a while, then you can see VSCode on browser.
- Open terminal on VSCode
Suppose `cargo` is installed.

Run Wasker
```
cargo run helloworld.wat
export LLVM_SYS_150_PREFIX=~/.wasker/clang+llvm-15.0.0-x86_64-linux-gnu-rhel-8.4/
cargo install wasker
```

# Run Wasker

## Option3 : Build from source
### Clone repository
```
git clone [email protected]:mewz-project/wasker.git
cd Wasker
```

### Install LLVM locally
The commands are a little different because you need an LLVM binary built for your architecture.
#### AMD64
```
mkdir -p dependencies/llvm
wget https://github.com/llvm/llvm-project/releases/download/llvmorg-15.0.0/clang+llvm-15.0.0-x86_64-linux-gnu-rhel-8.4.tar.xz -O /tmp/llvm-15.0.0.tar.xz
tar -xvf /tmp/llvm-15.0.0.tar.xz -C dependencies/llvm
export LLVM_SYS_150_PREFIX=$PWD/dependencies/llvm/clang+llvm-15.0.0-x86_64-linux-gnu-rhel-8.4
```
## Step1: Prepare Wasm binary
Please refer [examples](./examples) for building Wasm from Rust and Go.

#### AArch64
```
mkdir -p dependencies/llvm
wget https://github.com/llvm/llvm-project/releases/download/llvmorg-15.0.0/clang+llvm-15.0.0-aarch64-linux-gnu.tar.xz -O /tmp/llvm-15.0.0.tar.xz
tar -xvf /tmp/llvm-15.0.0.tar.xz -C dependencies/llvm
export LLVM_SYS_150_PREFIX=$PWD/dependencies/llvm/clang+llvm-15.0.0-aarch64-linux-gnu
cd examples/rust
rustup target add wasm32-wasi
cargo build --target wasm32-wasi
```

### Run Wasker
## Step2: Run Wasker
```
cargo run helloworld.wat
$ wasker examples/rust/target/wasm32-wasi/debug/rust.wasm
[2024-03-19T12:10:20Z INFO wasker::compiler] input: examples/rust/target/wasm32-wasi/debug/rust.wasm
[2024-03-19T12:10:20Z INFO wasker::compiler] write to ./wasm.ll
[2024-03-19T12:10:20Z INFO wasker::compiler] write to ./wasm.o, it may take a while
[2024-03-19T12:10:21Z INFO wasker::compiler] Compile success
```

# How to use Wasker outputs
## Step3: Run wasker output on Linux

ELF file generated by Wasker is OS-independent: WASI calls from Wasm applications remain unresolved.

Please write your own WASI wrapper for your favorite OS to be linked with Wasker output.

Here, we'll show a [tiny example](./examples/wasi-wrapper/wasi-wrapper-linux.c) of running Wasker output on Linux.

Link Wasker output and WASI wapper for Linux
Compile WASI wapper for Linux and ink with Wasker output.
```
gcc -no-pie ./examples/wasi-wrapper/wasi-wrapper-linux.c ./wasm.o -o hello
```
Expand All @@ -117,3 +71,26 @@ Run!!

Also please check [Mewz](https://github.com/Mewz-project/Mewz.git), a unikernel OS which has WASI interface.
ELF file generated by Wasker can be executed on Mewz without any modification.


# Development

## Option1 : Use Devcontainer
You can try Wasker via Devcontainer.

## Option2 : Build from source
```
git clone [email protected]:mewz-project/wasker.git
cd Wasker
export LLVM_SYS_150_PREFIX=~/.wasker/clang+llvm-15.0.0-x86_64-linux-gnu-rhel-8.4/
cargo install wasker
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
cargo install wasker
cargo build

```
By default, Wasker targets x86_64. If you want to target AArch64, please rewrite [build.rs](./build.rs) and override `LLVM_SYS_150_PREFIX` environment variable as follows.

```rust
# build.rs
5: let target = format!("clang+llvm-{}-aarch64-linux-gnu-rhel-8.4", llvm_version);

# shell
export LLVM_SYS_150_PREFIX=~/.wasker/clang+llvm-15.0.0-aarch64-linux-gnu/
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to automatically set the environment variable? By something like build.rs or cargo.toml?

https://stackoverflow.com/questions/57017066/how-do-i-set-environment-variables-with-cargo
https://doc.rust-lang.org/cargo/reference/build-scripts.html#rustc-env

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And it would be helpful if build.rs renames the LLVM directory to clang+llvm regardless of the platforms.

```
46 changes: 46 additions & 0 deletions build.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
use std::env;

fn main() {
let llvm_version = "15.0.0";
let target = format!("clang+llvm-{}-x86_64-linux-gnu-rhel-8.4", llvm_version);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could automatically detect the platform, and remove the need to edit it.

let home_dir = env::var("HOME").expect("fail to get home dir");
let wasker_dir = format!("{}/.wasker", home_dir);
let install_dir = format!("{}/{}", wasker_dir, target);

// Check if the specified version of LLVM is installed
if std::path::Path::new(&install_dir).exists() {
println!("LLVM is already installed in {}", install_dir);
return;
}
// Download LLVM from Github
println!("Downloading LLVM {}, it takes while...", llvm_version);
let url = format!(
"https://github.com/llvm/llvm-project/releases/download/llvmorg-{}/{}.tar.xz",
llvm_version, target
);
std::process::Command::new("wget")
.arg(&url)
.arg("-O")
.arg(format!("/tmp/{}.tar.xz", target))
.output()
.expect("Failed to download LLVM");

// Create directory
if !std::path::Path::new(&wasker_dir).exists() {
println!("Creating directory {:?}", wasker_dir);
std::fs::create_dir(&wasker_dir).expect("Failed to create directory");
}
Comment on lines +21 to +32
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could directly download the file, instead of using wget.


// Extract the tar file
println!("Extracting tar file to {:?}", wasker_dir);
std::process::Command::new("tar")
.arg("-xf")
.arg(format!("/tmp/{}.tar.xz", target))
.arg("-C")
.arg(wasker_dir)
.output()
.expect("Failed to extract tar file");
Comment on lines +35 to +42
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could directly extract tar files, instead of using tar command.


// Remove the tar file
std::fs::remove_file(format!("/tmp/{}.tar.xz", target)).expect("Failed to remove tar file");
}
Loading