Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 11 additions & 1 deletion docs/cn/server.md
Original file line number Diff line number Diff line change
Expand Up @@ -686,9 +686,19 @@ pthread模式可以让一些老代码快速尝试brpc,但我们仍然建议逐
- 设置内部端口。把ServerOptions.internal_port设为一个**仅允许内网访问**的端口。你可通过internal_port访问到内置服务,但通过对外端口(Server.Start时传入的那个)访问内置服务时将看到如下错误:

```
[a27eda84bcdeef529a76f22872b78305] Not allowed to access builtin services, try ServerOptions.internal_port=... instead if you're inside internal network
Not allowed to access builtin services, try ServerOptions.internal_port=... instead if you're inside internal network
```

反过来,internal_port只提供内置服务(以及Tabbed服务),普通服务的请求打到这个端口上会被拒绝:

```
Only builtin and Tabbed services are accessible on ServerOptions.internal_port=..., send the request to the port passed to Server::Start() instead
```

这是必须的:internal_port上的内置服务请求不需要通过ServerOptions.auth的鉴权,而鉴权结果是记在连接上的,一条连接只在第一个请求时鉴权一次。如果普通服务也在这个端口上提供,那么先发一个内置服务请求就能把整条连接标记为已鉴权,后续在同一条连接上访问普通服务将完全跳过鉴权。

有两类服务不走上面这条错误路径:ServerOptions.http_master_service和ServerOptions.baidu_master_service会接管所有URL和服务名(内置服务的也一样),它们在internal_port上被直接忽略,请求回退到正常的服务查找,这样内置服务仍然可以从这个端口访问;ServerOptions.redis_service在解析阶段就把命令处理完了,没有Controller可以带回EPERM,因此internal_port上干脆不提供redis协议,连接会被直接关闭。

- http proxy指定转发路径。nginx等可配置URL的映射关系,比如下面的配置把访问/MyAPI的外部流量映射到`target-server`的`/ServiceName/MethodName`。当外部流量尝试访问内置服务,比如说/status时,将直接被nginx拒绝。
```nginx
location /MyAPI {
Expand Down
12 changes: 11 additions & 1 deletion docs/en/server.md
Original file line number Diff line number Diff line change
Expand Up @@ -680,9 +680,19 @@ Builtin services are useful, on the other hand include a lot of internal informa
- Set internal port. Set ServerOptions.internal_port to a port which can **only be accessible from internal**. You can view builtin services via internal_port, while accesses from the public port (the one passed to Server.Start) should see following error:

```
[a27eda84bcdeef529a76f22872b78305] Not allowed to access builtin services, try ServerOptions.internal_port=... instead if you're inside internal network
Not allowed to access builtin services, try ServerOptions.internal_port=... instead if you're inside internal network
```

Conversely internal_port serves builtin (and Tabbed) services only, requests for ordinary services sent to it are rejected with:

```
Only builtin and Tabbed services are accessible on ServerOptions.internal_port=..., send the request to the port passed to Server::Start() instead
```

This is necessary: builtin requests on internal_port skip the authentication of ServerOptions.auth, and the verdict is remembered per connection since a connection is only authenticated once, on its first request. Were ordinary services served there as well, sending a builtin request first would mark the whole connection as authenticated and every later request on it would bypass authentication entirely.

Two kinds of services do not take the error path above. ServerOptions.http_master_service and ServerOptions.baidu_master_service answer for every URL and service name, the builtin ones included, so they are ignored on internal_port and the request falls back to the normal lookup, which keeps the builtin services reachable from that port. ServerOptions.redis_service handles the commands inside the parser and has no Controller to carry the EPERM, so internal_port does not speak redis at all and the connection is closed instead.

- http proxies only proxy specified URLs. nginx etc is able to configure how to map different URLs to back-end servers. For example the configure below maps public traffic to /MyAPI to `/ServiceName/MethodName` of `target-server`. If builtin services like /status are accessed from public, nginx rejects the attempts directly.
```nginx
location /MyAPI {
Expand Down
23 changes: 0 additions & 23 deletions src/brpc/details/server_private_accessor.h
Original file line number Diff line number Diff line change
Expand Up @@ -104,29 +104,6 @@ class ServerPrivateAccessor {
const Server* _server;
};

// Reject accesses to builtin services when the server is in security mode,
// in which case they are only reachable from ServerOptions.internal_port.
// Returns true if the access was rejected, in which case `cntl` was already
// SetFailed() and the caller must stop dispatching the request immediately.
// NOTE: Call this after ControllerPrivateAccessor::set_security_mode() and
// before the method is counted by MethodStatus::OnRequested(), so that
// rejected accesses do not pollute the stats of the method. `mp` may point
// to BadMethodService which is builtin as well and lists the methods of the
// requested service, so protocols dispatching to BadMethodService must call
// this beforehand, or make sure the listing is hidden in security mode.
inline bool RejectBuiltinAccess(Controller* cntl, const Server& server,
const Server::MethodProperty* mp) {
if (!cntl->is_security_mode() ||
(!mp->is_builtin_service && !mp->params.is_tabbed)) {
return false;
}
cntl->SetFailed(EPERM, "Not allowed to access builtin services, try "
"ServerOptions.internal_port=%d instead if you're in "
"internal network",
server.options().internal_port);
return true;
}

// Count one error if release() is not called before destruction of this object.
class ScopedNonServiceError {
public:
Expand Down
17 changes: 9 additions & 8 deletions src/brpc/nshead_pb_service_adaptor.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -114,22 +114,23 @@ void NsheadPbServiceAdaptor::ProcessNsheadRequest(
}

ServerPrivateAccessor server_accessor(&server);
const Server::MethodProperty *sp = server_accessor
const Server::MethodProperty* mp = server_accessor
.FindMethodPropertyByFullName(meta->full_method_name());
if (nullptr == sp ||
sp->service->GetDescriptor() == BadMethodService::descriptor()) {
if (nullptr == mp ||
mp->service->GetDescriptor() == BadMethodService::descriptor()) {
controller->SetFailed(ENOMETHOD, "Fail to find method=%s",
meta->full_method_name().c_str());
break;
}
if (RejectBuiltinAccess(controller, server, sp)) {
if (server.RejectBuiltinAccess(controller, mp) ||
server.RejectNonBuiltinAccessFromInternalPort(controller, mp)) {
break;
}
pbdone->status = sp->status;
sp->status->OnRequested();
pbdone->status = mp->status;
mp->status->OnRequested();

google::protobuf::Service* svc = sp->service;
const google::protobuf::MethodDescriptor* method = sp->method;
google::protobuf::Service* svc = mp->service;
const google::protobuf::MethodDescriptor* method = mp->method;
ControllerPrivateAccessor(controller).set_method(method);
done->SetMethodName(butil::EnsureString(method->full_name()));
pbdone->pbreq.reset(svc->GetRequestPrototype(method).New());
Expand Down
23 changes: 15 additions & 8 deletions src/brpc/policy/baidu_rpc_protocol.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -278,6 +278,12 @@ struct BaiduProxyPBMessages : public RpcPBMessages {
};
}

static bool IsBaiduMasterService(const Server* server,
const butil::EndPoint& local_side) {
return nullptr != server->options().baidu_master_service &&
!server->IsInternalPort(local_side);
}

// Used by UT, can't be static.
void SendRpcResponse(int64_t correlation_id, Controller* cntl,
RpcPBMessages* messages, const Server* server,
Expand Down Expand Up @@ -306,10 +312,10 @@ void SendRpcResponse(int64_t correlation_id, Controller* cntl,
}

cntl->CallAfterRpcResp(req, res);
if (nullptr == server->options().baidu_master_service) {
server->options().rpc_pb_message_factory->Return(messages);
} else {
if (IsBaiduMasterService(server, cntl->local_side())) {
BaiduProxyPBMessages::Return(static_cast<BaiduProxyPBMessages*>(messages));
} else {
server->options().rpc_pb_message_factory->Return(messages);
}
};

Expand Down Expand Up @@ -713,13 +719,13 @@ void ProcessRpcRequest(InputMessageBase* msg_base) {

google::protobuf::Service* svc = nullptr;
google::protobuf::MethodDescriptor* method = nullptr;
if (nullptr != server->options().baidu_master_service) {
if (IsBaiduMasterService(server, cntl->local_side())) {
if (socket->is_overcrowded() &&
!server->options().ignore_eovercrowded &&
!server->options().baidu_master_service->ignore_eovercrowded()) {
cntl->SetFailed(EOVERCROWDED, "Connection to %s is overcrowded",
butil::endpoint2str(socket->remote_side()).c_str());
break;
cntl->SetFailed(EOVERCROWDED, "Connection to %s is overcrowded",
butil::endpoint2str(socket->remote_side()).c_str());
break;
}
svc = server->options().baidu_master_service;
auto sampled_request = new SampledRequest;
Expand Down Expand Up @@ -770,7 +776,8 @@ void ProcessRpcRequest(InputMessageBase* msg_base) {
request_meta.method_name().c_str());
break;
}
if (RejectBuiltinAccess(cntl.get(), *server, mp)) {
if (server->RejectBuiltinAccess(cntl.get(), mp) ||
server->RejectNonBuiltinAccessFromInternalPort(cntl.get(), mp)) {
break;
}
if (mp->service->GetDescriptor() == BadMethodService::descriptor()) {
Expand Down
6 changes: 4 additions & 2 deletions src/brpc/policy/http_rpc_protocol.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -1558,7 +1558,8 @@ void ProcessHttpRequest(InputMessageBase *msg) {
return;
}

if (server->options().http_master_service) {
if (server->options().http_master_service &&
!server->IsInternalPort(cntl->local_side())) {
// If http_master_service is on, just call it.
google::protobuf::Service* svc = server->options().http_master_service;
const google::protobuf::MethodDescriptor* md =
Expand Down Expand Up @@ -1605,7 +1606,8 @@ void ProcessHttpRequest(InputMessageBase *msg) {
mp->service->CallMethod(mp->method, cntl, &breq, &bres, nullptr);
return;
}
if (RejectBuiltinAccess(cntl, *server, mp)) {
if (server->RejectBuiltinAccess(cntl, mp) ||
Comment thread
chenBright marked this conversation as resolved.
server->RejectNonBuiltinAccessFromInternalPort(cntl, mp)) {
return;
}
// Switch to service-specific error.
Expand Down
19 changes: 10 additions & 9 deletions src/brpc/policy/hulu_pbrpc_protocol.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -442,36 +442,37 @@ void ProcessHuluRequest(InputMessageBase* msg_base) {
break;
}

const Server::MethodProperty *sp =
const Server::MethodProperty* mp =
server_accessor.FindMethodPropertyByNameAndIndex(
meta.service_name(), meta.method_index());
if (nullptr == sp) {
if (nullptr == mp) {
cntl->SetFailed(ENOMETHOD, "Fail to find method=%d of service=%s",
meta.method_index(), meta.service_name().c_str());
break;
}
if (RejectBuiltinAccess(cntl.get(), *server, sp)) {
if (server->RejectBuiltinAccess(cntl.get(), mp) ||
server->RejectNonBuiltinAccessFromInternalPort(cntl.get(), mp)) {
break;
}
if (sp->service->GetDescriptor() == BadMethodService::descriptor()) {
if (mp->service->GetDescriptor() == BadMethodService::descriptor()) {
BadMethodRequest breq;
BadMethodResponse bres;
breq.set_service_name(meta.service_name());
sp->service->CallMethod(sp->method, cntl.get(), &breq, &bres, nullptr);
mp->service->CallMethod(mp->method, cntl.get(), &breq, &bres, nullptr);
break;
}
if (socket->is_overcrowded() &&
!server->options().ignore_eovercrowded &&
!sp->ignore_eovercrowded) {
!mp->ignore_eovercrowded) {
cntl->SetFailed(EOVERCROWDED, "Connection to %s is overcrowded",
butil::endpoint2str(socket->remote_side()).c_str());
break;
}

// Switch to service-specific error.
non_service_error.release();
method_status = sp->status;
const google::protobuf::MethodDescriptor* method = sp->method;
method_status = mp->status;
const google::protobuf::MethodDescriptor* method = mp->method;
const std::string method_full_name = butil::EnsureString(method->full_name());
if (method_status) {
int rejected_cc = 0;
Expand All @@ -482,7 +483,7 @@ void ProcessHuluRequest(InputMessageBase* msg_base) {
}
}

google::protobuf::Service* svc = sp->service;
google::protobuf::Service* svc = mp->service;
accessor.set_method(method);

if (!server->AcceptRequest(cntl.get())) {
Expand Down
3 changes: 3 additions & 0 deletions src/brpc/policy/mongo_protocol.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -252,6 +252,9 @@ void ProcessMongoRequest(InputMessageBase* msg_base) {
mongo_done->cntl.SetFailed(ENOMETHOD, "Fail to find default_method");
break;
}
if (server->RejectNonBuiltinAccessFromInternalPort(&mongo_done->cntl, mp)) {
break;
}
// Switch to service-specific error.
non_service_error.release();
MethodStatus* method_status = mp->status;
Expand Down
3 changes: 3 additions & 0 deletions src/brpc/policy/nshead_protocol.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -314,6 +314,9 @@ void ProcessNsheadRequest(InputMessageBase* msg_base) {
cntl->SetFailed(ELOGOFF, "Server is stopping");
break;
}
if (server->RejectNonBuiltinAccessFromInternalPort(cntl)) {
break;
}
if (socket->is_overcrowded() && !server->options().ignore_eovercrowded) {
cntl->SetFailed(EOVERCROWDED, "Connection to %s is overcrowded",
butil::endpoint2str(socket->remote_side()).c_str());
Expand Down
8 changes: 8 additions & 0 deletions src/brpc/policy/redis_protocol.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -125,6 +125,14 @@ ParseResult ParseRedisMessage(butil::IOBuf* source, Socket* socket,
if (!rs) {
return MakeParseError(PARSE_ERROR_TRY_OTHERS);
}
if (server->IsInternalPort(socket->local_side())) {
// ServerOptions.internal_port serves builtin and Tabbed services
// only and a RedisService is neither. The command handlers run
// right here rather than in ProcessRedisRequest(), there is no
// Controller to reject the request with, so behave as if redis
// was not enabled on this port at all.
return MakeParseError(PARSE_ERROR_TRY_OTHERS);
}
RedisConnContext* ctx = static_cast<RedisConnContext*>(socket->parsing_context());
if (ctx == nullptr) {
ctx = new RedisConnContext(rs);
Expand Down
17 changes: 9 additions & 8 deletions src/brpc/policy/sofa_pbrpc_protocol.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -403,36 +403,37 @@ void ProcessSofaRequest(InputMessageBase* msg_base) {
break;
}

const Server::MethodProperty *sp =
const Server::MethodProperty* mp =
server_accessor.FindMethodPropertyByFullName(meta.method());
if (nullptr == sp) {
if (nullptr == mp) {
cntl->SetFailed(ENOMETHOD, "Fail to find method=%s",
meta.method().c_str());
break;
}
if (RejectBuiltinAccess(cntl.get(), *server, sp)) {
if (server->RejectBuiltinAccess(cntl.get(), mp) ||
server->RejectNonBuiltinAccessFromInternalPort(cntl.get(), mp)) {
break;
}
if (socket->is_overcrowded() &&
!server->options().ignore_eovercrowded &&
!sp->ignore_eovercrowded) {
!mp->ignore_eovercrowded) {
cntl->SetFailed(EOVERCROWDED, "Connection to %s is overcrowded",
butil::endpoint2str(socket->remote_side()).c_str());
break;
}
// Switch to service-specific error.
non_service_error.release();
method_status = sp->status;
method_status = mp->status;
if (method_status) {
int rejected_cc = 0;
if (!method_status->OnRequested(&rejected_cc)) {
cntl->SetFailed(ELIMIT, "Rejected by %s's ConcurrencyLimiter, concurrency=%d",
butil::EnsureString(sp->method->full_name()).c_str(), rejected_cc);
butil::EnsureString(mp->method->full_name()).c_str(), rejected_cc);
break;
}
}
google::protobuf::Service* svc = sp->service;
const google::protobuf::MethodDescriptor* method = sp->method;
google::protobuf::Service* svc = mp->service;
const google::protobuf::MethodDescriptor* method = mp->method;
accessor.set_method(method);

if (!server->AcceptRequest(cntl.get())) {
Expand Down
3 changes: 3 additions & 0 deletions src/brpc/policy/thrift_protocol.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -510,6 +510,9 @@ void ProcessThriftRequest(InputMessageBase* msg_base) {
" ServerOptions.thrift_service, close the connection.";
return cntl->SetFailed(EINTERNAL, "ServerOptions.thrift_service is NULL");
}
if (server->RejectNonBuiltinAccessFromInternalPort(cntl)) {
return;
}

// Switch to service-specific error.
non_service_error.release();
Expand Down
36 changes: 36 additions & 0 deletions src/brpc/server.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -2371,6 +2371,42 @@ bool Server::AcceptRequest(Controller* cntl) const {
return true;
}

bool Server::RejectBuiltinAccess(Controller* cntl,
const MethodProperty* mp) const {
if (!cntl->is_security_mode() ||
(!mp->is_builtin_service && !mp->params.is_tabbed)) {
return false;
}
cntl->SetFailed(EPERM, "Not allowed to access builtin services, try "
"ServerOptions.internal_port=%d instead if you're in internal network",
_options.internal_port);
return true;
}

bool Server::RejectNonBuiltinAccessFromInternalPort(
Controller* cntl, const MethodProperty* mp) const {
if (mp->is_builtin_service || mp->params.is_tabbed) {
return false;
}
return RejectNonBuiltinAccessFromInternalPort(cntl);
}

bool Server::RejectNonBuiltinAccessFromInternalPort(Controller* cntl) const {
if (!IsInternalPort(cntl->local_side())) {
return false;
}
cntl->SetFailed(EPERM, "Only builtin and Tabbed services are accessible on "
"ServerOptions.internal_port=%d, send the request to the port "
"passed to Server::Start() instead",
_options.internal_port);
return true;
}

bool Server::IsInternalPort(const butil::EndPoint& local_side) const {
return _options.internal_port >= 0 &&
local_side.port == _options.internal_port;
}

#ifdef SSL_CTRL_SET_TLSEXT_HOSTNAME
int Server::SSLSwitchCTXByHostname(struct ssl_st* ssl,
int* al, void* se) {
Expand Down
Loading
Loading