Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add support for array types and more functions #2

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from
Draft

Conversation

abhizer
Copy link
Collaborator

@abhizer abhizer commented Jan 2, 2025

adds support for testing array types and more functions that are available in feldera

@abhizer abhizer requested a review from mihaibudiu January 2, 2025 09:13
@abhizer abhizer marked this pull request as draft January 2, 2025 13:31
Copy link

@mihaibudiu mihaibudiu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can add methods for generating random values to the compiler if you can reuse the data types there. We have literals for all these types - what you call constants.

import java.util.stream.Collectors;

import com.yugabyte.jdbc.EscapeSyntaxCallMode;

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is this needed?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is not needed, I didn't even notice that.
Will remove.

case DOUBLE:
case TINYINT:
case SMALLINT:
case FLOAT:

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why this change?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I changed this type to only represent the primitive types, so FLOAT represents DOUBLE, BOOL, and INT represents all the different sizes of the integer types.
This is similar to the implementation done by other databases in sqlancer.

@@ -121,6 +81,31 @@ public static FelderaDataType getRandomNonNullType() {
public static FelderaDataType getRandomType() {
return Randomly.fromOptions(values());
}
}

public static class FelderaCompositeDataType {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is not general enough, MAP has two generic arguments.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe call this GenericType?
Too bad we aren't reusing the structures we already have in the compiler.

@@ -129,29 +114,144 @@ public FelderaExpression getRandomConstant(FelderaGlobalState globalState) {

return FelderaConstant.getRandomConstant(globalState, this);
}

public boolean isNumeric() {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks wrong.
An array is not numeric.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This method is not specific to arrays.
It returns true for int types, FP types and decimal.

return this.elementType;
}

public static FelderaCompositeDataType getBooleanType() {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand what this class is supposed to be.
Maybe it's a type factory?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this code should be in a separate factory class, which may also implement a singleton pattern.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is designed represents all datatypes in feldera (doesn't quite support map yet).
Similar to other implementations:

public static class CockroachDBCompositeDataType {

}

@Override
public String toString() {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this suggests that Type should be an interface and these all subclasses.
What are the constraints imposed from SqlLancer?
Maybe you really should reuse the compiler type hierarchy.

@@ -13,30 +13,32 @@ public class FelderaAggregate implements FelderaExpression {
private boolean blackbox;

public enum FelderaAggregateFunction {
AVG(FelderaSchema.FelderaDataType.INT, FelderaSchema.FelderaDataType.DOUBLE),
AVG(FelderaSchema.FelderaDataType.INT, FelderaSchema.FelderaDataType.FLOAT),

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we don't even support FLOAT in SQL, this is very confusing.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With FLOAT, we represent both DOUBLE and REAL here.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Many functions work on double but not on real.

case NULL:
return new FelderaNullConstant();
case TIME:
return FelderaTimeConstant.getRandom(globalState);
return new FelderaCast(FelderaTimeConstant.getRandom(globalState), type);

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this because now time literals are strings?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes.

@mihaibudiu
Copy link

How about I try to rework this code to reuse the compiler ir? We should pack the ir in a separate jar and publish it too.

@abhizer
Copy link
Collaborator Author

abhizer commented Jan 4, 2025

How about I try to rework this code to reuse the compiler ir? We should pack the ir in a separate jar and publish it too.

I think that would be nice. I wanted to do that, but couldn't figure out how to do it nicely.

* disables NATURAL join for now as it produces a lot of compilation
  errors

Signed-off-by: Abhinav Gyawali <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants