Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FDK testkit initial commit #1

Open
wants to merge 16 commits into
base: master
Choose a base branch
from
Open

FDK testkit initial commit #1

wants to merge 16 commits into from

Conversation

denismakogon
Copy link
Member

No description provided.

@denismakogon
Copy link
Member Author

@treeder @zootalures new repo, new PR instead of fnproject/fn#549

@denismakogon denismakogon changed the title Fdk FDK testkit initial commit Dec 5, 2017
@treeder
Copy link

treeder commented Dec 5, 2017

Can you not check in vendor dir? Makes PR's a disaster.

@denismakogon
Copy link
Member Author

@treeder Out of curiosity, why?

 HTTP 501 means that FDK does not support specific
 format, so that's not a reason to fail test.
@denismakogon
Copy link
Member Author

@treeder So, I tried to put deps into its own commit, so, I'm sorry if that looks like a mess, but I do recommend to review by commits.

@denismakogon
Copy link
Member Author

From @rdallman
Compliance criteria:

  • runs > 1 call in same image [hot], serially
  • [eventually] times out a call properly (i.e. takes input of next call when expected)
  • parses headers for the set of expected headers, e.g. can output correct CALL_ID on N runs
  • certain vars accessible from env (make sure no masking of env)
  • supports default and >= 1 hot format based on FN_FORMAT env var changing

ideally, the code looks a certain way, but this requires humans or so-much-static-analysis-it's-not-worth-it in 2017. (i.e. want to make sure they have a 'context' variable that has config, headers on it)

@rdallman
Copy link

rdallman commented Dec 5, 2017

tiny addendum to 1) / possibly separate test than serial test:

  • assert that input changes output (e.g. force them to 'echo' input to output) -- if we run with random numbers as input they cannot game this.

formats_test.go Outdated
wg.Add(times)

for i := 0; i < times; i++ {
go func() {
Copy link
Member

@zootalures zootalures Dec 6, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CallMultiple should do a sequential test, not a a parallel test

ideally it would be good to have some witness that multiple events were streamed to to the same container? but I don' think there is a way to safely guarantee this.

The purpose of this test is to verify that the FDK can handle a sequence of events to the same container (i think)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

addressed

utils.go Outdated
}

func Cleanup() {
ctx := context.Background()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be useful to optionally keep the app (e.g. via env) so that I can get logs for failures.

utils.go Outdated
EnvAsHeader(req, env)
}

resp, err := http.DefaultClient.Do(req)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be useful to log the app/callId here

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

got it

formats_test.go Outdated
t.Errorf("Output assertion error.\n\tExpected: %v\n\tActual: %v", expected, output.String())
}
if response.StatusCode != http.StatusOK {
if response.StatusCode == http.StatusNotImplemented {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not clear how an FDK that doesn't understand a specific format can indicate that via a 501 right now (as the status code has to be encoded in the specified format)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be better to let the tested formats be specified via env? :
e.g. :

// FilterTestedFormats filters a list of formats to those that are tested for this FDK, based on a comma-separated list in the "FDK_FORMATS" environment variable
func FilterTestedFormats(formats []string )[]string {
	supportedFormats := os.Getenv("FDK_FORMATS")

	if supportedFormats != "" {
		acceptedFormats := strings.Split(supportedFormats,",")
		validFormats := []string{}

		for _,af := range acceptedFormats {
			for _, reqF := range formats {
				if reqF == af {
					validFormats = append(validFormats,reqF)
				}
			}
		}
		return validFormats

	}
	return formats
}


       formats := FilterTestedFormats([]string{"http", "json"})

	helloJohnPayload := &struct {
		Name string `json:"name"`
	}{}
       ... 

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, minor thing, but don't have objections.

@denismakogon
Copy link
Member Author

@treeder good news, vendor has gone (easy to review and build).

Dockerfile Outdated
RUN apk --no-cache add build-base git bzr mercurial gcc
RUN go get -u github.com/golang/dep/cmd/dep
ENV D=/go/src/github.com/fnproject/fdk-testkit
ADD . $D
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See my pull in fn core, should gopy only gopkg files, then do dep ensure --vendor-only, then do this add in a layer after that.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

addressed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants