Allow line-log circling

[typeless]

If the length of the log exceeds the limit, drop the oldest lines to fill the new ones.

[CLAassistant]

All committers have signed the CLA.

[tboerger]

Ok, so it's a ring buffer for the logs, right?

[typeless]

@tboerger Yes

[bradrydzewski]

one issue is that this will not place any restrictions on the size of the logs that are streamed (in real time) to the server which stores the stream in memory. Storing the stream in memory (for dozens or even hundreds of concurrent builds) is not an issue because the size of the logs are limited (e.g. 100 concurrent builds cannot use more than 500mb ram on the server). When this limit is removed it presents an opportunity to overload the server memory.

[typeless]

@bradrydzewski As I understand it, every time append needs more space than the slice current has, it will allocate a new one and copy the content of the old slice to it. Then the memory chunk of the old slice can be recycled. The lines (pointers) not copied to the new space are dead and can be recycled too. So the cap of the slice keeps under the limit.
https://play.golang.org/p/-3SZNMLs_5h
https://github.com/golang/go/blob/master/src/runtime/slice.go#L76-L191

That said, there might be something I am missing, and only actual measurement can make sure.

[typeless]

I did some simplified test:

mura@bs003:~/tmp$ cat a.go
package main

import (
        "fmt"
)

func main() {
        myslice := make([]byte, 1024)
        for i := 0; i < 1000000000; i++ {
                myslice = append(myslice, 'A')
                myslice = myslice[1:]

        }
        fmt.Printf("cap(myslice):%d len(myslice):%d [0]:%c [-1]:%c\n", cap(myslice), len(myslice), myslice[0], myslice[len(myslice)-1])

}
mura@bs003:~/tmp$ go build a.go
mura@bs003:~/tmp$ /usr/bin/time -v ./a
cap(myslice):1024 len(myslice):1024 [0]:A [-1]:A
        Command being timed: "./a"
        User time (seconds): 6.72
        System time (seconds): 0.26
        Percent of CPU this job got: 132%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 0:05.28
        Average shared text size (kbytes): 0
        Average unshared data size (kbytes): 0
        Average stack size (kbytes): 0
        Average total size (kbytes): 0
        Maximum resident set size (kbytes): 6668
        Average resident set size (kbytes): 0
        Major (requiring I/O) page faults: 0
        Minor (reclaiming a frame) page faults: 9844
        Voluntary context switches: 21049
        Involuntary context switches: 13
        Swaps: 0
        File system inputs: 0
        File system outputs: 0
        Socket messages sent: 0
        Socket messages received: 0
        Signals delivered: 0
        Page size (bytes): 4096
        Exit status: 0

As expected, the peak memory does not grow to 1GB.

[bradrydzewski]

that is testing the agent. I am speaking of the server-side package that stores the streams in-memory at https://github.com/drone/drone/tree/master/livelog.

The circular buffer does not restrict the size of logs pushed to the server (this line) and therefore allows unbounded log lines to accumulate. In 2013 (in a very early prototype of Drone) a single build generated 120mb log file and took down our server. With no limit to the stream, the server would be susceptible to this issue.

[typeless]

I understand what you mean now.

[typeless]

Just had a look at it.
It seems that the stream logger has done something similar
(https://github.com/drone/drone/blob/master/livelog/stream.go#L43-L57) to avoid unbounded log lines.
About the channel subscribers, I can find only one in the live_log test.
https://github.com/drone/drone/blob/master/livelog/livelog.go#L79
I am not 100% confident, though.

[tboerger]

Even if this implementation is not final now, a ring buffer to get the last parts of the log makes really sense.

[dschmidt]

Any news on any way forward?