Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

netty: Per-rpc call option authority verification against peer cert subject names #11724

Open
wants to merge 36 commits into
base: master
Choose a base branch
from
Open

netty: Per-rpc call option authority verification against peer cert subject names #11724

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
fa36f83
In-progress changes.
kannanjgithub Nov 18, 2024
f31b8bc
In-progress changes that used ExtendedSSLSession.
kannanjgithub Nov 26, 2024
0152478
Authority verify in Netty transport.
kannanjgithub Nov 27, 2024
5e2e22e
Netty authority verify against peer host in server cert.
kannanjgithub Nov 30, 2024
b0f86cf
unit test.
kannanjgithub Dec 1, 2024
5ba5431
nit changes and drop unintended changes.
kannanjgithub Dec 2, 2024
e42492b
nit changes and drop unintended changes.
kannanjgithub Dec 2, 2024
5285353
Cache the peer verification result.
kannanjgithub Dec 4, 2024
ea969ef
Move extraction of X509ExtendedTrustManager to utils.
kannanjgithub Dec 4, 2024
94172de
Fixes.
kannanjgithub Dec 4, 2024
0ddd42c
Fixes.
kannanjgithub Dec 4, 2024
5109115
Fixes.
kannanjgithub Dec 4, 2024
d5f3968
nit
kannanjgithub Dec 4, 2024
1e072d4
In-progress review comments.
kannanjgithub Dec 9, 2024
32104ce
Address review comments.
kannanjgithub Dec 9, 2024
95aae4a
revert unintended
kannanjgithub Dec 9, 2024
c0327b5
revert unintended
kannanjgithub Dec 9, 2024
b24a605
Address review comments.
kannanjgithub Dec 13, 2024
862b95b
Fix warning.
kannanjgithub Dec 13, 2024
8902dbd
Address review comments.
kannanjgithub Dec 14, 2024
01b0eb2
Address review comments.
kannanjgithub Dec 14, 2024
8dd8749
Remove duplicate definitions of createTrustManager.
kannanjgithub Dec 14, 2024
3d744d9
Review comments.
kannanjgithub Dec 16, 2024
4ef0fdb
Move NoopSslSession to io.grpc.internal under grpc-core project.
kannanjgithub Dec 18, 2024
fdc6e94
Added flag with default false for the per-rpc authority check.
kannanjgithub Dec 19, 2024
60efd84
Fix style.
kannanjgithub Jan 7, 2025
d2af807
Merge branch 'master' into authoritychecktls
kannanjgithub Jan 7, 2025
2594842
Fix merge conflicts.
kannanjgithub Jan 7, 2025
916d0d5
Review comments and use reflection for X509ExtendedTrustManager.
kannanjgithub Jan 10, 2025
040035b
Include failed method name in the tls verification failed log message.
kannanjgithub Jan 13, 2025
44f2412
Address review comments.
kannanjgithub Jan 15, 2025
6449728
Address review comments.
kannanjgithub Jan 16, 2025
4273452
Address review comments.
kannanjgithub Jan 17, 2025
a9a019b
Remove the code handling for impossible exception.
kannanjgithub Jan 17, 2025
1f56282
Update comment for NoSuchMethodError.
kannanjgithub Jan 17, 2025
b8b70bf
Merge branch 'master' into authoritychecktls
kannanjgithub Jan 21, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion netty/src/main/java/io/grpc/netty/NettyClientTransport.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -197,7 +197,11 @@ public ClientStream newStream(
if (callOptions.getAuthority() != null) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see now that we're getting this per-RPC authority separately from the data flow that uses the authority. That makes me nervous, especially for security. As the code changes, it's likely this will break. In fact, it is already broken because the LB-based override only influences setAuthority().

For using the authority, the per-RPC authority is copied to the stream in ClientCallImpl. NettyClientStream then copies it into the Netty Http2Headers (while running an application's thread).

Having the logic in NettyClientTransport has been a bit odd, but fine, and I saw why it was done to avoid plumbing of the protocol negotiator to other places. Normally we'd add RPC logic in NettyClientHandler/NettyClientStream. NettyClientTransport doesn't do much other than create/manage the Netty channel (connection). I suggest we move this logic to NettyClientHandler and forgo any cache synchronization because it will always run on the transport thread.

Status verificationStatus = negotiator.verifyAuthority(callOptions.getAuthority());
if (!verificationStatus.isOk()) {
return new FailingClientStream(verificationStatus, tracers);
if (GrpcUtil.getFlag("GRPC_ENABLE_PER_RPC_AUTHORITY_CHECK", false)) {
ejona86 marked this conversation as resolved.
Show resolved Hide resolved
return new FailingClientStream(verificationStatus, tracers);
}
channelLogger.log(ChannelLogger.ChannelLogLevel.WARNING, "Authority '{}' specified via call" +
ejona86 marked this conversation as resolved.
Show resolved Hide resolved
" options for rpc did not match peer certificate's subject names.");
}
}
StatsTraceContext statsTraceCtx =
Expand Down
135 changes: 85 additions & 50 deletions netty/src/test/java/io/grpc/netty/NettyClientTransportTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -842,64 +842,99 @@ public void tlsNegotiationServerExecutorShouldSucceed() throws Exception {
@Test
public void authorityOverrideInCallOptions_noX509ExtendedTrustManager_newStreamCreationFails()
throws IOException, InterruptedException, GeneralSecurityException {
startServer();
InputStream caCert = TlsTesting.loadCert("ca.pem");
X509TrustManager x509ExtendedTrustManager =
(X509TrustManager) getX509ExtendedTrustManager(caCert).get();
ProtocolNegotiators.FromChannelCredentialsResult result =
ProtocolNegotiators.from(TlsChannelCredentials.newBuilder()
.trustManager(new FakeTrustManager(x509ExtendedTrustManager)).build());
NettyClientTransport transport = newTransport(result.negotiator.newNegotiator());
FakeClientTransportListener fakeClientTransportListener = new FakeClientTransportListener();
callMeMaybe(transport.start(fakeClientTransportListener));
synchronized (fakeClientTransportListener) {
fakeClientTransportListener.wait(10000);
System.setProperty("GRPC_ENABLE_PER_RPC_AUTHORITY_CHECK", "true");
try {
startServer();
InputStream caCert = TlsTesting.loadCert("ca.pem");
X509TrustManager x509ExtendedTrustManager =
(X509TrustManager) getX509ExtendedTrustManager(caCert).get();
ProtocolNegotiators.FromChannelCredentialsResult result =
ProtocolNegotiators.from(TlsChannelCredentials.newBuilder()
.trustManager(new FakeTrustManager(x509ExtendedTrustManager)).build());
NettyClientTransport transport = newTransport(result.negotiator.newNegotiator());
FakeClientTransportListener fakeClientTransportListener = new FakeClientTransportListener();
callMeMaybe(transport.start(fakeClientTransportListener));
synchronized (fakeClientTransportListener) {
fakeClientTransportListener.wait(10000);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wait is allowed to spuriously return early, which means this could be flaky. It also looks like isConnected should be @GuardedBy("this") and all reads/writes to it happen within the lock, since you can't guarantee the wakeup was because of the notify().

Dealing with the spurious wakeup is slightly annoying when there's a timeout. I suggest using a Future<Void> connected = SettableFuture.create() and completing the future instead of setting isConnected to true. Or use a CountDownLatch.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

}
assertThat(fakeClientTransportListener.isConnected).isTrue();

ClientStream stream = transport.newStream(
Rpc.METHOD, new Metadata(), CallOptions.DEFAULT.withAuthority("foo.test.google.in"),
new ClientStreamTracer[]{new ClientStreamTracer() {
}});

assertThat(stream).isInstanceOf(FailingClientStream.class);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The instanceOf is a bit of a code smell, but within acceptability, but abusing the timeout insight is not great. The nice opaque/black-box testing here is to pass a Listener (a mock is fine) to stream.start() and see that the stream fails (closed() is called, and you can see the passed status).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

InsightBuilder insightBuilder = new InsightBuilder();
stream.appendTimeoutInsight(insightBuilder);
assertThat(insightBuilder.toString()).contains(
"Status{code=FAILED_PRECONDITION, description=Can't allow authority override in rpc when "
+ "SslEngine or X509ExtendedTrustManager is not available, cause=null}");
} finally {
System.clearProperty("GRPC_ENABLE_PER_RPC_AUTHORITY_CHECK");
}
assertThat(fakeClientTransportListener.isConnected).isTrue();

ClientStream stream = transport.newStream(
Rpc.METHOD, new Metadata(), CallOptions.DEFAULT.withAuthority("foo.test.google.in"),
new ClientStreamTracer[]{new ClientStreamTracer() {
}});

assertThat(stream).isInstanceOf(FailingClientStream.class);
InsightBuilder insightBuilder = new InsightBuilder();
stream.appendTimeoutInsight(insightBuilder);
assertThat(insightBuilder.toString()).contains(
"Status{code=FAILED_PRECONDITION, description=Can't allow authority override in rpc when "
+ "SslEngine or X509ExtendedTrustManager is not available, cause=null}");
}

@Test
public void authorityOverrideInCallOptions_doesntMatchServerPeerHost_newStreamCreationFails()
throws IOException, InterruptedException, GeneralSecurityException {
startServer();
NettyClientTransport transport = newTransport(newNegotiator());
FakeClientTransportListener fakeClientTransportListener = new FakeClientTransportListener();
callMeMaybe(transport.start(fakeClientTransportListener));
synchronized (fakeClientTransportListener) {
fakeClientTransportListener.wait(10000);
System.setProperty("GRPC_ENABLE_PER_RPC_AUTHORITY_CHECK", "true");
try {
startServer();
NettyClientTransport transport = newTransport(newNegotiator());
FakeClientTransportListener fakeClientTransportListener = new FakeClientTransportListener();
callMeMaybe(transport.start(fakeClientTransportListener));
synchronized (fakeClientTransportListener) {
fakeClientTransportListener.wait(10000);
}
assertThat(fakeClientTransportListener.isConnected).isTrue();

ClientStream stream = transport.newStream(
Rpc.METHOD, new Metadata(), CallOptions.DEFAULT.withAuthority("foo.test.google.in"),
new ClientStreamTracer[]{new ClientStreamTracer() {
}});

assertThat(stream).isInstanceOf(FailingClientStream.class);
InsightBuilder insightBuilder = new InsightBuilder();
stream.appendTimeoutInsight(insightBuilder);
assertThat(insightBuilder.toString()).contains(
"Status{code=UNAVAILABLE, description=Peer hostname verification during rpc failed for"
+ " authority 'foo.test.google.in'");
assertThat(insightBuilder.toString()).contains("cause=java.security.cert.CertificateException:"
+ " No subject alternative DNS name matching foo.test.google.in found.");
} finally {
System.clearProperty("GRPC_ENABLE_PER_RPC_AUTHORITY_CHECK");
}
assertThat(fakeClientTransportListener.isConnected).isTrue();

ClientStream stream = transport.newStream(
Rpc.METHOD, new Metadata(), CallOptions.DEFAULT.withAuthority("foo.test.google.in"),
new ClientStreamTracer[]{new ClientStreamTracer() {
}});

assertThat(stream).isInstanceOf(FailingClientStream.class);
InsightBuilder insightBuilder = new InsightBuilder();
stream.appendTimeoutInsight(insightBuilder);
assertThat(insightBuilder.toString()).contains(
"Status{code=UNAVAILABLE, description=Peer hostname verification during rpc failed for"
+ " authority 'foo.test.google.in'");
assertThat(insightBuilder.toString()).contains("cause=java.security.cert.CertificateException:"
+ " No subject alternative DNS name matching foo.test.google.in found.");
}

@Test
public void authorityOverrideInCallOptions_matchesServerPeerHost_newStreamCreationSucceeds()
throws IOException, InterruptedException, GeneralSecurityException {
System.setProperty("GRPC_ENABLE_PER_RPC_AUTHORITY_CHECK", "true");
try {
startServer();
NettyClientTransport transport = newTransport(newNegotiator());
FakeClientTransportListener fakeClientTransportListener = new FakeClientTransportListener();
callMeMaybe(transport.start(fakeClientTransportListener));
synchronized (fakeClientTransportListener) {
fakeClientTransportListener.wait(10000);
}
assertThat(fakeClientTransportListener.isConnected).isTrue();

ClientStream stream = transport.newStream(
Rpc.METHOD, new Metadata(), CallOptions.DEFAULT.withAuthority("zoo.test.google.fr"),
new ClientStreamTracer[]{new ClientStreamTracer() {
}});

assertThat(stream).isNotInstanceOf(FailingClientStream.class);
} finally {
System.clearProperty("GRPC_ENABLE_PER_RPC_AUTHORITY_CHECK");
}
}

@Test
public void authorityOverrideInCallOptions_notMatches_flagDisabled_createsStream()
throws IOException, InterruptedException, GeneralSecurityException {
startServer();
NettyClientTransport transport = newTransport(newNegotiator());
FakeClientTransportListener fakeClientTransportListener = new FakeClientTransportListener();
Expand All @@ -910,11 +945,11 @@ public void authorityOverrideInCallOptions_matchesServerPeerHost_newStreamCreati
assertThat(fakeClientTransportListener.isConnected).isTrue();

ClientStream stream = transport.newStream(
Rpc.METHOD, new Metadata(), CallOptions.DEFAULT.withAuthority("zoo.test.google.fr"),
new ClientStreamTracer[]{new ClientStreamTracer() {
}});
Rpc.METHOD, new Metadata(), CallOptions.DEFAULT.withAuthority("foo.test.google.in"),
new ClientStreamTracer[]{new ClientStreamTracer() {
}});

assertThat(stream).isNotInstanceOf(FailingClientStream.class);
assertThat(stream).isInstanceOf(NettyClientStream.class);
}

private Throwable getRootCause(Throwable t) {
Expand Down
Loading