Skip to content

Commit

Permalink
Incorporate reviewer feedback.
Browse files Browse the repository at this point in the history
  • Loading branch information
fniephaus committed Sep 26, 2024
1 parent d6470c2 commit 022f567
Show file tree
Hide file tree
Showing 2 changed files with 26 additions and 17 deletions.
14 changes: 10 additions & 4 deletions graalpy/graalpy-openai-starter/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ To build and test the demo, run:
To execute the main method, run:

```bash
./mvnw exec:java
./mvnw exec:java -Dexec.args="'Say Hello from GraalPy'"
```

## Run the Application Using Gradle
Expand All @@ -39,18 +39,24 @@ To build and test the demo, run:
To execute the main method, run:

```bash
./gradlew run
./gradlew run --args="'Say Hello from GraalPy'"
```

## Implementation Details

The [App.java](src/main/java/com/example/App.java) embeds [the official usage example](https://github.com/openai/openai-python?tab=readme-ov-file#usage), which requires that an OpenAI API key is set via the `OPENAI_API_KEY` environment variable.
The [App.java](src/main/java/com/example/App.java) embeds a `create_chat_completion()` function based on [the official usage example](https://github.com/openai/openai-python?tab=readme-ov-file#usage).
The Java text block that contains the Python code is annotated with `// language=python`, which triggers a [language injection in IntelliJ IDEA](https://www.jetbrains.com/help/idea/using-language-injections.html).
The `create_chat_completion()` function is returned when the Python code is evaluated.
Afterward, the Python function is called with `userInput` from Java, and the result is mapped to the `ChatCompletion` interface.
The interfaces `ChatCompletion`, `Choice`, and `ChatComplectionMessage` are ported from [the official API](https://github.com/openai/openai-python/blob/main/api.md), and only contain the functionality exercised by the example code.
These interfaces can be extended appropriately when needed.

Also, note that the [_pom.xml_](pom.xml) contains the `openai` package and all its transitive dependencies, and locks their version.
Moreover, the [_pom.xml_](pom.xml) contains the `openai` package and all its transitive dependencies, and locks their version.
This is recommended because it provides a similar experience to Maven (all versions are well-defined), while `pip`, the package installer for Python, also supports version ranges (versions are defined at compile time).
In particular, the `jiter` and `pydantic_core` make use of native extensions which must be built for GraalPy.
For the selected versions of these two packages, there are pre-built binary wheels for Linux x86_64 in the [GraalPy PyPI repository](https://www.graalvm.org/python/wheels/).
This means that on Linux x86_64 machines, the native extensions do not need to be built from source and are instead fetched from the PyPI repository.

> Note: Although `jiter`, `pydantic_core`, and many other Python packages that use native extensions work with GraalPy, their use is currently considered experimental.
> Also note that when using multiple contexts, only one context can load and access a native extension at a time.
> We are exploring approaches to support multi-context setups with native extensions.
29 changes: 16 additions & 13 deletions graalpy/graalpy-openai-starter/src/main/java/com/example/App.java
Original file line number Diff line number Diff line change
Expand Up @@ -8,15 +8,17 @@

import org.graalvm.polyglot.Context;
import org.graalvm.polyglot.PolyglotException;
import org.graalvm.polyglot.Value;
import org.graalvm.python.embedding.utils.GraalPyResources;

import java.util.List;

public class App {

public static void main(String[] args) {
String userInput = args.length > 0 ? args[0] : "Say this is a test";
try (Context context = GraalPyResources.createContext()) {
ChatCompletion chatCompletion = context.eval("python",
Value createChatCompletion = context.eval("python",
// language=python
"""
import os
Expand All @@ -27,19 +29,20 @@ public static void main(String[] args) {
api_key=os.environ.get("OPENAI_API_KEY"),
)
chat_completion = client.chat.completions.create(
messages=[
{
"role": "user",
"content": "Say this is a test",
}
],
model="gpt-3.5-turbo",
)
chat_completion
""").as(ChatCompletion.class);
def create_chat_completion(user_input):
return client.chat.completions.create(
messages=[
{
"role": "user",
"content": user_input,
}
],
model="gpt-3.5-turbo",
)
create_chat_completion
""");
ChatCompletion chatCompletion = createChatCompletion.execute(userInput).as(ChatCompletion.class);
for (Choice choice : chatCompletion.choices()) {
System.out.println(choice.message().content());
}
Expand Down

0 comments on commit 022f567

Please sign in to comment.