JZTXT

django 结合rpc服务传输

发布时间 2023-10-12 20:15:32作者: Οo白麒麟оΟ

概述

gRPC（gRPC Remote Procedure Call）是一种高性能、跨语言的远程过程调用（RPC）框架，由Google开发并开源。它基于HTTP/2协议进行通信，使用Protocol Buffers（protobuf）作为默认的消息编码格式。gRPC的目标是提供简单的、高效的RPC服务，使不同应用程序、服务或组件能够跨网络进行通信，而不必担心底层通信细节。

gRPC 使用 Protocol Buffers 来定义消息和服务，这意味着在通信中使用了严格的消息类型，确保了数据的一致性和完整性。

如果使用的话，是否会对当前的随意扩展的服务带来一定的限制。

基础依赖

 
pip install grpcio pip install grpcio-tools

定义服务和消息

Language Guide (proto 3)

eg：

 
syntax = "proto3"; package job; service JobService { rpc GetJob(GetJobRequest) returns (GetJobResponse) {} } message GetJobRequest { int32 id = 1; } // 1,2 tag message Location { int32 id = 1; string value = 2; int32 region_id = 3; string region = 4; bool is_qualified = 5; } message Skill { string value = 1; bool is_qualified = 5; } message GetJobResponse { int32 id = 1; string job_title = 2; int32 job_type_id = 3; string job_type_value = 4; string job_location = 5; Location location = 6; repeated Skill skills = 7; }

生成 gRPC 代码

 
python -m grpc_tools.protoc -I./protos --python_out=. --pyi_out=. --grpc_python_out=. ./protos/job.proto

或者在相关服务端和客户端导入模块

 
job_pb2, job_pb2_grpc = grpc.protos_and_services("job.proto")

protos_and_services 函数会动态地加载 .proto 文件，运行时生成相应的Python类和方法

在一些场景中，可能更倾向于提前生成这些文件，以便能够更清晰地查看和理解由 .proto 文件生成的代码，或者因为开发、构建或部署流程更适合使用静态生成的文件。

两者之间的选择取决于具体需求和开发流程。

如果觉得动态加载更方便，可以使用grpc.protos_and_services

如果希望能够直接查看和管理生成的Python代码，可以使用grpcio-tools生成对应的文件

创建服务和客户端服务

Server

 
import os from concurrent import futures import logging import grpc from rest_framework.generics import get_object_or_404 from rpc_services import job_pb2, job_pb2_grpc from jobs.models import Job from jobs.serializers import JobDetailSerializer classJobGRPCService(job_pb2_grpc.JobService): defGetJob(self, request, context): job_id = request.idprint(job_id)# job = get_object_or_404(# Job,# pk=job_id,# )# print(job)# serializer = JobDetailSerializer(# job, context={"jobseeker": None}# )# data = serializer.data data ={"id":167267,"job_title":"Java Developer","job_type_id":1,"job_type_value":"Full-time","job_location":"bishan","location":{"id":162,"value":"Bishan","region_id":21,"region":"Central","is_qualified":False},"xp_lvl":{"id":2,"key":"1_to_3_years","value":"1 - 3 Yrs Exp","is_qualified":False},"skills":[{"value":"Python","is_qualified":False},{"value":"Java","is_qualified":False}]}print(data)# ValueError: Protocol message GetJobResponse has no "xp_lvl" field.""" int32 id = 1; string job_title = 2; int32 job_type_id = 3; string job_type_value = 4; string job_location = 5; Location location = 6; repeated Skill skills = 7; """return job_pb2.GetJobResponse(id=data["id"], job_title=data["job_title"], job_type_id=data["job_type_id"], job_type_value=data["job_type_value"], job_location=data["job_location"], location=data["location"], skills=data["skills"],) defserve():print("Starting serve") logging.basicConfig() port ="50051" server = grpc.server(futures.ThreadPoolExecutor(max_workers=10)) job_pb2_grpc.add_JobServiceServicer_to_server(JobGRPCService(), server) server.add_insecure_port("[::]:"+ port) server.start()print("Server started, listening on "+ port) server.wait_for_termination() if __name__ =="__main__":# logging.basicConfig() serve()

Client

 
from __future__ import print_function import logging import grpc import job_pb2 import job_pb2_grpc from google.protobuf.json_format import MessageToJson defrun():# NOTE(gRPC Python Team): .close() is possible on a channel and should be# used in circumstances in which the with statement does not fit the needs# of the code.print("Will try to get job ...")with grpc.insecure_channel("localhost:50051")as channel: stub = job_pb2_grpc.JobServiceStub(channel) response = stub.GetJob(job_pb2.GetJobRequest(id=167267)) print(f"Client received: {response}") d = MessageToJson(response)print(d) if __name__ =="__main__": logging.basicConfig() run()

启动服务端和客户端

 
python server.py python client.py

Django中集成gRPC

可以使用 django-grpc-framework

AWS Lambda部署存在一定困难 应该无法依赖WSGI运行

通过Django Command 部署运行

独立rpc服务，通过容器镜像封装

安全认证方面

 

https://grpc.io/docs/guides/auth/

 

基于 TLS/SSL 的证书认证

基于令牌的认证

HTTP 标头认证

eg:

 
token ="token"metadata =[('authorization','Bearer '+ token)]response = stub.GetJob(job_pb2.GetJobRequest(id=167267), metadata=metadata)

 

健康检测

 

https://github.com/grpc/grpc/blob/master/examples/python/health_checking/greeter_server.py

 

相关文档生成

 

https://github.com/pseudomuto/protoc-gen-doc