Invalid characters in terminal after special characters were displayed during prebuild

[shaal]

Bug description

Invalid characters in terminal after special characters were displayed during prebuild.

In the screenshot above -
Aftermc (Minio client) was running during prebuild (these are the green characters),
A new workspace opens with invalid characters in terminal (ie. 11;rgb:1e1e/1e1e/1e1e;1R)

I'm sure it's not related specifically to mc, but has to do with special characters that are displayed during prebuild.
But mc was the easiest example I could find to reproduce the problem I'm seeing.

Steps to reproduce

1. Open the example repository in Gitpod - https://github.com/shaal/gitpod-test--invalid-characters-in-terminal
2. When workspace open - confirm you see invalid characters in terminal

Workspace affected

No response

Expected behavior

Regardless of what special characters are displayed during prebuild, it should not affect the terminal in a new workspace.
Especially when these invalid characters are "typed" in the command line.

Example repository

https://github.com/shaal/gitpod-test--invalid-characters-in-terminal

Anything else?

No response

[loujaybee]

Thanks @shaal ! We'll take a look 🙏

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[abitrolly]

I've got the same problem in this PR https://github.com/sourcegraph/sourcegraph/pull/43684

# Sourcegraph quickStart: https://docs.sourcegraph.com/dev/setup/quickstart
# Gitpod config reference: https://www.gitpod.io/docs/config-gitpod-file

tasks:
  - name: sg
    init: |
      (
        set -e

        ./dev/sg/bootstrap.sh -f -p=false
      )
    command: |
      (
        set -e
    
        echo "Run 'sg setup' to setup development environment"
        echo "https://docs.sourcegraph.com/dev/setup/quickstart"
      )        

Can we reopen this issue?

[abitrolly]

@axonasif I am not sure it is a good-first-issue, because the cause is still unknown, and it is not clear where to look for it.

I can't remember cases in other CI/CD systems where output from previous commands affected input of the next one without explicit pipe.

[axonasif]

@abitrolly I thought it could be a good-first-issue, it could be fixed by issuing the task commands differently.

[abitrolly]

As a workaround maybe, but not the solution that would keep users happy (speaking of myself, of course).

So do you know anybody who could explain why this happens?

[axonasif]

As a workaround maybe, but not the solution that would keep users happy (speaking of myself, of course).

My bad, maybe I wasn't clear. I mean from Gitpod side.

Here's the task terminal creation logic:
https://github.com/gitpod-io/gitpod/blob/main/components/supervisor/pkg/supervisor/tasks.go

[abitrolly]

Not sure I can understand what's going on just by looking at that file. This init method. Is it the code that processes init: task?

gitpod/components/supervisor/pkg/supervisor/tasks.go

Line 172 in 1573864

func (tm *tasksManager) init(ctx context.Context) {

[jeanp413]

I dug a little deeper and I'm almost sure it's an issue with the golang pty library, something like this issue (though that is macos related)

[jeanp413]

Update:

• First issue, the missing emoji in the case of sourcegraph example, was because the pty was started with 0 rows and columns, needed to set a default rows and columns from the start, draft PR fixing it Set default cols and rows when creating pty #15143
• Second issue, invalid characters 11;rgb:1e1e/1e1e/1e1e;1R shown in terminal, in the case of sourcegraph example it's because the sourcegraph cli is using termenv library and internally it's sending some special vt sequences to the terminal, specifically this one echo -e "\x1b]11;?\x07"; is used to query the background color and 11;rgb:1e1e/1e1e/1e1e;1R is the response to that, problem is that our pty is without a client so there's no one to interpret those sequences (those are usually handled by the terminal UI) and they are stored as is in the buffer, later when a client connects, the buffer gets send and printed to the terminal UI which interprets the vt sequences and prints the response.
  To fix it we need to interpret those vt sequences even without a client, I think vscode do this when restoring a terminal session, internally it uses xterm.js in headless mode, maybe we can use https://github.com/hinshun/vt10x for golang

Test to repro (sourcegraph binary can be downloaded from here)

func TestBug(t *testing.T) {
	var (
		terminals = NewMux()
	)

	alias, err := terminals.Start(exec.Command("/usr/bin/bash"), TermOptions{
		ReadTimeout: 0,
	})
	if err != nil {
		t.Fatal(err)
	}
	term, ok := terminals.Get(alias)
	if !ok {
		t.Fatal("terminal is not found")
	}

	// _, err = term.PTY.Write([]byte("./sg_linux_amd64 install -f -p=false; exit \n"))
	_, err = term.PTY.Write([]byte("echo -e \"\\x1b]11;?\\x07\"; exit \n"))
	if err != nil {
		t.Fatal(err)
	}

	stdout := term.Stdout.Listen()
	buf := new(strings.Builder)
	_, err = io.Copy(buf, stdout)
	if err != nil {
		t.Fatal(err)
	}
	fmt.Println(">>>>>>>>>>>>>")
	var str = buf.String()
	fmt.Println(str)
	fmt.Println("<<<<<<<<<<<<<")
}