nova api执行过程(以nova list为例)
2016-10-13 18:44
99 查看
nova api执行过程(以nova list为例)
本文仅供学习参考:使用devstack部署的openstack,需了解如何在screen中重新启动一个服务
请注意时间,因为openstack处在不断更新中,本文学习的是当前最新的openstack版本
在screen下用ipdb进行调试
对openstack整个架构有个大致了解
没有写明根目录的默认根目录是/opt/stack/nova(即openstack源码目录下的nova目录)
REQ: curl -g -i -X GET http://10.238.158.189/identity -H “Accept: application/json” -H “User-Agent: nova keystoneauth1/2.12.1 python-requests/2.11.1 CPython/2.7.6”
REQ: curl -g -i -X GET http://10.238.158.189:8774/v2.1 -H “User-Agent: python-novaclient” -H “Accept: application/json” -H “X-Auth-Token: {SHA1}1ed8ae56fa11986c2d3aef62a58d3355d9178d2c”
REQ: curl -g -i -X GET http://10.238.158.189:8774/v2.1/servers/detail -H “OpenStack-API-Version: compute 2.37” -H “User-Agent: python-novaclient” -H “Accept: application/json” -H “X-OpenStack-Nova-API-Version: 2.37” -H “X-Auth-Token: {SHA1}1ed8ae56fa11986c2d3aef62a58d3355d9178d2c”
第一步:从Keystone拿到一个授权的token
第二步:把返回的token填入api请求中,该步为验证版本v2.1之类的(不确定)
第三步:同样需要将token填入api请求,这一个步骤是真正的主要的请求即获得所有当前活跃状态的虚拟机
nova list命令转化为HTTP请求
由novaclient实现,不关注…HTTP请求到WSGI Application
这里需要了解下WSGI和paste配置文件,WSGI推荐 –> https://www.fullstackpython.com/wsgi-servers.htmlNova API服务nova-api(在screen中对应的编号为6的window)在启动 时(没有发出HTTP请求时),会根据Nova配置文件(/etc/nova/nova.conf)的enable_apis选项内容创建一个或多个WSGI Server,用devstack部署默认的配置如下
enabled_apis = osapi_compute,metadata
Paste Deploy会在各个WSGI Server创建使参与进来,基于Paste配置文件(/etc/nova/api-paste.ini)去加载WSGI Application,加载WSGI Application由nova/service.py实现,加载后,WSGI Application就处在等待和监听状态。
class WSGIService(service.Service): """Provides ability to launch API from a 'paste' configuration.""" def __init__(self, name, loader=None, use_ssl=False, max_url_len=None): """Initialize, but do not start the WSGI server. :param name: The name of the WSGI server given to the loader. :param loader: Loads the WSGI application using the given name. :returns: None """ self.name = name # 这里的name就是WSGI server name,比如osapi_compute或者metadata # nova.service's enabled_apis self.binary = 'nova-%s' % name self.topic = None self.manager = self._get_manag 4000 er() self.loader = loader or wsgi.Loader() # 从paste配置文件加载 Nova API 对应的 WSGI Application self.app = self.loader.load_app(name) # inherit all compute_api worker counts from osapi_compute if name.startswith('openstack_compute_api'): wname = 'osapi_compute' else: wname = name self.host = getattr(CONF, '%s_listen' % name, "0.0.0.0") self.port = getattr(CONF, '%s_listen_port' % name, 0) self.workers = (getattr(CONF, '%s_workers' % wname, None) or processutils.get_worker_count()) if self.workers and self.workers < 1: worker_name = '%s_workers' % name msg = (_("%(worker_name)s value of %(workers)s is invalid, " "must be greater than 0") % {'worker_name': worker_name, 'workers': str(self.workers)}) raise exception.InvalidInput(msg) self.use_ssl = use_ssl self.server = wsgi.Server(name, self.app, host=self.host, port=self.port, use_ssl=self.use_ssl, max_url_len=max_url_len) # Pull back actual port used self.port = self.server.port self.backdoor_port = None
在nova-api运行过程中(发出了HTTP请求),Paste Deploy会将WSGI Server上监听到的HTTP请求根据Paste配置文件准确地路由到特定的WSGI Application,这其中经过了
nova.api.openstack.urlmap的urlmap_factory
nova.api.auth的pipeline_factory_v21
nova.api.openstack.compute的APIRouterV21.factory(路由到指定的app)
下面根据nova –debug list输出的请求信息,一一介绍
GET http://10.238.158.189:8774/v2.1/servers/detail
1.首先我们通过这个请求知道使用了v2.1的API,再根据paste配置文件中
[composite:osapi_compute] <-- use = call:nova.api.openstack.urlmap:urlmap_factory <-- /: oscomputeversions /v2: openstack_compute_api_v21_legacy_v2_compatible /v2.1: openstack_compute_api_v21 <--
则WSGI服务器osapi_compute将监听到这个请求,根据第一行use的内容我们知道是由nova.api.openstack.urlmap模块的urlmap_factory进行分发,即对应了openstack_compute_api_v21
2.在paste配置文件中,又有
[composite:openstack_compute_api_v21] <-- use = call:nova.api.auth:pipeline_factory_v21 <-- noauth2 = cors http_proxy_to_wsgi compute_req_id faultwrap sizelimit noauth2 osapi_compute_app_v21 keystone = cors http_proxy_to_wsgi compute_req_id faultwrap sizelimit authtoken keystonecontext osapi_compute_app_v21 <--
可知又使用了nova.api.auth模块的pipeline_factory_v21进一步分发,并根据/etc/nova/nova.conf中认证策略“auth_strategy”选项使用参数keystone,在paste文件中,我们看到noauth2和keystone后面跟着一大串,不用过于深究,这是一个pipeline,最后一个是真正的app即osapi_compute_app_v21,从代码中分析可知根据这个pipeline,从后往前为这个app穿上一件件“外衣”,每一次封装都是封装成一个WSGI Application。
代码实现见nova/api/auth.py
def _load_pipeline(loader, pipeline): filters = [loader.get_filter(n) for n in pipeline[:-1]] app = loader.get_app(pipeline[-1]) filters.reverse() for filter in filters: app = filter(app) return app
3.再由paste配置文件中
[app:osapi_compute_app_v21] paste.app_factory = nova.api.openstack.compute:APIRouterV21.factory
需要调用nova.api.openstack.compute的APIRouterV21.factory
WSGI Application到具体的执行函数
这个部分是最复杂的部分,函数的调用和封装很复杂,有时候看到最深层次也看不大懂了,如果有愿意分享的小伙伴欢迎多多交流APIRouterV21类的定义在nova/api/openstack/__init__.py中,主要完成对所有资源的加载和路由规则的创建(得到一个mapper包含所有资源的路由信息),APIRouterV21初始化的最后一步是将这个mapper交给它的父类nova.wsgi.Router,完成mapper和_dismatch()的关联,在openstack中,每个资源对应一个Controller,也对应一个WSGI Application,比如请求
GET http://10.238.158.189:8774/v2.1/servers/detail
对应的Contrller就是资源servers的Controller,具体位置是nova/api/openstack/compute/servers.py中的ServersController,这个Controller中定义了各种action,其中有一个函数是detail(),最终会调用这个函数得到想要的结果
下面详细介绍其过程
1.所有资源的加载以及路由规则的创建
class APIRouterV21(base_wsgi.Router): def __init__(self, init_only=None): def _check_load_extension(ext): return self._register_extension(ext) # 使用stevedore的EnabledExtensionManager类载入位于setup.cfg # 的nova.api.v21.extensions的所有资源 self.api_extension_manager = stevedore.enabled.EnabledExtensionManager( namespace=self.api_extension_namespace(), check_func=_check_load_extension, invoke_on_load=True, invoke_kwds={"extension_info": self.loaded_extension_info}) mapper = ProjectMapper() self.resources = {} # NOTE(cyeoh) Core API support is rewritten as extensions # but conceptually still have core if list(self.api_extension_manager): # NOTE(cyeoh): Stevedore raises an exception if there are # no plugins detected. I wonder if this is a bug. # 所有资源进行注册,同时使用mapper建立路由规则 self._register_resources_check_inherits(mapper) # 扩展现有资源及其操作 self.api_extension_manager.map(self._register_controllers) LOG.info(_LI("Loaded extensions: %s"), sorted(self.loaded_extension_info.get_extensions().keys())) super(APIRouterV21, self).__init__(mapper)
2.父类Router将mapper和dispatch()关联起来
class Router(object): """WSGI middleware that maps incoming requests to WSGI apps.""" def __init__(self, mapper): """Create a router for the given routes.Mapper. Each route in `mapper` must specify a 'controller', which is a WSGI app to call. You'll probably want to specify an 'action' as well and have your controller be an object that can route the request to the action-specific method. Examples: mapper = routes.Mapper() sc = ServerController() # Explicit mapping of one route to a controller+action mapper.connect(None, '/svrlist', controller=sc, action='list') # Actions are all implicitly defined mapper.resource('server', 'servers', controller=sc) # Pointing to an arbitrary WSGI app. You can specify the # {path_info:.*} parameter so the target app can be handed just that # section of the URL. mapper.connect(None, '/v1.0/{path_info:.*}', controller=BlogApp()) """ self.map = mapper self._router = routes.middleware.RoutesMiddleware(self._dispatch, self.map) # <-- @webob.dec.wsgify(RequestClass=Request) def __call__(self, req): """Route the incoming request to a controller based on self.map. If no match, return a 404. """ return self._router @staticmethod @webob.dec.wsgify(RequestClass=Request) def _dispatch(req): """Dispatch the request to the appropriate controller. Called by self._router after matching the incoming request to a route and putting the information into req.environ. Either returns 404 or the routed WSGI app's response. """ match = req.environ['wsgiorg.routing_args'][1] if not match: return webob.exc.HTTPNotFound() app = match['controller'] return app
这里的__call__函数进行了wsgify的封装,在这个Router被call的时候,或者是它的子类APIRouterV21在被call的时候,需要看webob.dec.wsgify里面是怎么进行封装的才能知道具体的执行过程。我们已经了解到,在从HTTP请求路由到特定的WSGI Apllication的时候,最终路由到的app是经过封装的,即经过了pipeline列出的一系列filter,所以这个时候我们要真正地调用时,需要一层层把外面的filter去掉,这个在/usr/local/lib/python2.7/dist-packages/webob/dec.py中的wsgify类中有一个__call__的定义
def __call__(self, req, *args, **kw): """Call this as a WSGI application or with a request""" func = self.func if func is None: if args or kw: raise TypeError( "Unbound %s can only be called with the function it " "will wrap" % self.__class__.__name__) func = req return self.clone(func) if isinstance(req, dict): if len(args) != 1 or kw: raise TypeError( "Calling %r as a WSGI app with the wrong signature") environ = req start_response = args[0] req = self.RequestClass(environ) req.response = req.ResponseClass() try: args = self.args if self.middleware_wraps: args = (self.middleware_wraps,) + args resp = self.call_func(req, *args, **self.kwargs) except HTTPException as exc: resp = exc if resp is None: ## FIXME: I'm not sure what this should be? resp = req.response if isinstance(resp, text_type): resp = bytes_(resp, req.charset) if isinstance(resp, bytes): body = resp resp = req.response resp.write(body) if resp is not req.response: resp = req.response.merge_cookies(resp) return resp(environ, start_response) else: if self.middleware_wraps: args = (self.middleware_wraps,) + args return self.func(req, *args, **kw)
这个函数定义了,被dec.wsgify封装后的函数在被调用的时候,会不断地把外面的“外衣”脱掉,一直到最核心的app,这个app是一个直接接收参数(environ, start_response)的wsg
aec9
i app,我们看到在Router类的__init__操作中,初始化了一个self.router = routes.middleware.RoutesMiddleware(self._dispatch,self.map),而routes.middleware.RoutesMiddleware的__call_函数是一个标准的WSGI Application,它接收参数(environ, start_response),如下,
def __call__(self, environ, start_response): """Resolves the URL in PATH_INFO, and uses wsgi.routing_args to pass on URL resolver results.""" old_method = None if self.use_method_override: req = None #(以下省略...)
所以在APIRouter被call的整个过程中,我们可以通过在dec.wsgify的__call__函数中加打印信息查看整个流程,打印每次被call时的func信息,结果如下(总共有三次类似的输出,因为有三次请求)
******************************************************************************** <bound method CORS.__call__ of <oslo_middleware.cors.CORS object at 0x7f15db89ef50>> ******************************************************************************** <bound method HTTPProxyToWSGI.__call__ of <oslo_middleware.http_proxy_to_wsgi.HTTPProxyToWSGI object at 0x7f15db89eed0>> ******************************************************************************** <bound method ComputeReqIdMiddleware.__call__ of <nova.api.compute_req_id.ComputeReqIdMiddleware object at 0x7f15db89ee50>> ******************************************************************************** <bound method FaultWrapper.__call__ of <nova.api.openstack.FaultWrapper object at 0x7f15db89ec50>> ******************************************************************************** <bound method RequestBodySizeLimiter.__call__ of <oslo_middleware.sizelimit.RequestBodySizeLimiter object at 0x7f15db89edd0>> ******************************************************************************** <bound method AuthProtocol.__call__ of <keystonemiddleware.auth_token.AuthProtocol object at 0x7f15db89e690>> ******************************************************************************** <bound method NovaKeystoneContext.__call__ of <nova.api.auth.NovaKeystoneContext object at 0x7f15db89e6d0>> ******************************************************************************** <bound method APIRouterV21.__call__ of <nova.api.openstack.compute.APIRouterV21 object at 0x7f15dbf45990>>
和paste配置文件中的pipeline的顺序是一样的,即
keystone = cors http_proxy_to_wsgi compute_req_id faultwrap sizelimit authtoken keystonecontext osapi_compute_app_v21
小结
看了几天的代码,很多还是一头雾水,python调用的库很多,openstack里又经常进行封装操作,有时候一个调用栈非常深,很容易混乱,这里涉及到的比如mapper是怎么建立路由的,等等,都没有在此深究,这种先看官网的exsample再尝试看代码,多设断点多调试,调试技巧很重要。继续努力!有小伙伴感兴趣的欢迎讨论。相关文章推荐
- [nova]nova api执行过程分析
- Nova API的执行过程
- nova hypervisor-list无法执行,其他api均正常
- openstack nova 源码解析 — Nova API 执行过程从(novaclient到Action)
- salt的api学习记录--salt命令的执行过程
- (坑!!待补)Linux____基础API底层执行过程(网络通信)
- 利用mfc类和odbcapi执行存储过程
- socket从userspace到kernel的api执行过程(不含tcp/ip协议栈部分)
- linux 之 基础API底层执行过程
- salt的api学习记录---salt-cp命令的执行过程
- nova list的nova-api.log
- 关于ODBC api 执行sqlserver存储过程的问题
- nova-api对web请求的路由过程的分析
- Liberty nova-api HTTP请求执行流程
- Nova启动虚拟机执行过程
- C#执行存储过程的简化
- CPU执行机器代码的过程
- 用存储过程执行Insert和直接执行Insert的性能比较
- 执行存储过程的多种写法
- [FAQ]ASP中提取多选列表框中的值,并传递给存储过程执行。