apache mesos 入门(4)——mesos中的资源
2016-04-22 17:11
731 查看
mesos根据角色(ROLE)在不同的framework之间分配资源,在启动master时可以指定系统中的角色和每种角色的权重,例如:
mesos-master --ip=127.0.0.1 --work_dir=/var/lib/mesos --roles=qarole,devrole --weights='qarole=1,devrole=3'
指定了两种角色parole和devrole,权重分别是1和3.
在设置FrameworkInfo时可以设置对应的角色
slave启动时会检测当前节点的资源情况,包括cpu、内存、磁盘等,slave默认会预留1G(或者50%)的内存,5G(或者50%)的磁盘以保证自己及操作系统的正常运行,剩下的资源会上报master以供调度。当然也可以手动指定应该上报的资源量,例如,
mesos-slave --master=127.0.0.1:5050 --resources='cpus:4;mem:8192;ports:[1000-5000,31000-32000];'
slave还可以静态的设置各个角色的资源量,例如:
mesos-slave --master=127.0.0.1:5050 --resources='cpus(prod):1;mem(prod):4096;cpus(qa):2;mem(qa):1024;cpus:13;mem:11264' --attributes='operating_system:ubuntu;cpu_class:haswell;zone:us_east;rack:22'
其中prod角色:1cpu、4096内存
qa角色:2cpu、1024内存
默认角色:13cpu、11264内存
除了上报slave节点的资源情况,也可以标记当前节点的一些属性以供scheduler使用,比如是否ssd硬盘,操作系统类型等
mesos-slave --master=127.0.0.1:5050 --resources='cpus:4;mem:8192;ports:[1000-5000,31000-32000];' --attributes='operating_system:ubuntu;cpu_class:haswell;zone:us_east;rack:22'
mesos-master --ip=127.0.0.1 --work_dir=/var/lib/mesos --roles=qarole,devrole --weights='qarole=1,devrole=3'
指定了两种角色parole和devrole,权重分别是1和3.
在设置FrameworkInfo时可以设置对应的角色
/** * Describes a framework. */ message FrameworkInfo { // Used to determine the Unix user that an executor or task should // be launched as. If the user field is set to an empty string Mesos // will automagically set it to the current user. required string user = 1; // Name of the framework that shows up in the Mesos Web UI. required string name = 2; // Note that 'id' is only available after a framework has // registered, however, it is included here in order to facilitate // scheduler failover (i.e., if it is set then the // MesosSchedulerDriver expects the scheduler is performing // failover). optional FrameworkID id = 3; // The amount of time (in seconds) that the master will wait for the // scheduler to failover before it tears down the framework by // killing all its tasks/executors. This should be non-zero if a // framework expects to reconnect after a failure and not lose its // tasks/executors. // // NOTE: To avoid accidental destruction of tasks, production // frameworks typically set this to a large value (e.g., 1 week). optional double failover_timeout = 4 [default = 0.0]; // If set, framework pid, executor pids and status updates are // checkpointed to disk by the slaves. Checkpointing allows a // restarted slave to reconnect with old executors and recover // status updates, at the cost of disk I/O. optional bool checkpoint = 5 [default = false]; <span style="color:#ff0000;"> // Used to group frameworks for allocation decisions, depending on // the allocation policy being used. optional string role = 6 [default = "*"];</span> // Used to indicate the current host from which the scheduler is // registered in the Mesos Web UI. If set to an empty string Mesos // will automagically set it to the current hostname if one is // available. optional string hostname = 7; // This field should match the credential's principal the framework // uses for authentication. This field is used for framework API // rate limiting and dynamic reservations. It should be set even // if authentication is not enabled if these features are desired. optional string principal = 8; // This field allows a framework to advertise its web UI, so that // the Mesos web UI can link to it. It is expected to be a full URL, // for example http://my-scheduler.example.com:8080/. optional string webui_url = 9; message Capability { enum Type { // Receive offers with revocable resources. See 'Resource' // message for details. // TODO(vinod): This is currently a no-op. REVOCABLE_RESOURCES = 1; // Receive the TASK_KILLING TaskState when a task is being // killed by an executor. The executor will examine this // capability to determine whether it can send TASK_KILLING. TASK_KILLING_STATE = 2; } required Type type = 1; } // This field allows a framework to advertise its set of // capabilities (e.g., ability to receive offers for revocable // resources). repeated Capability capabilities = 10; // Labels are free-form key value pairs supplied by the framework // scheduler (e.g., to describe additional functionality offered by // the framework). These labels are not interpreted by Mesos itself. // Labels should not contain duplicate key-value pairs. optional Labels labels = 11; }
slave启动时会检测当前节点的资源情况,包括cpu、内存、磁盘等,slave默认会预留1G(或者50%)的内存,5G(或者50%)的磁盘以保证自己及操作系统的正常运行,剩下的资源会上报master以供调度。当然也可以手动指定应该上报的资源量,例如,
mesos-slave --master=127.0.0.1:5050 --resources='cpus:4;mem:8192;ports:[1000-5000,31000-32000];'
slave还可以静态的设置各个角色的资源量,例如:
mesos-slave --master=127.0.0.1:5050 --resources='cpus(prod):1;mem(prod):4096;cpus(qa):2;mem(qa):1024;cpus:13;mem:11264' --attributes='operating_system:ubuntu;cpu_class:haswell;zone:us_east;rack:22'
其中prod角色:1cpu、4096内存
qa角色:2cpu、1024内存
默认角色:13cpu、11264内存
除了上报slave节点的资源情况,也可以标记当前节点的一些属性以供scheduler使用,比如是否ssd硬盘,操作系统类型等
mesos-slave --master=127.0.0.1:5050 --resources='cpus:4;mem:8192;ports:[1000-5000,31000-32000];' --attributes='operating_system:ubuntu;cpu_class:haswell;zone:us_east;rack:22'
相关文章推荐
- 安装xampp后,出现“Apache 2 Test Page powered by CentOS“
- Apache端口映射
- apache mesos 入门(3)——自定义framework + executor
- win10 占用80端口 导致 wamp apache服务不能正常启动
- Apache Karaf离线打包
- java.lang.NoClassDefFoundError: org/apache/log4j/spi/NOPLoggerRepository
- apache commons fileupload 问题跟踪系统
- ERROR org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode: Exception in doCheckpoint
- java.lang.NoClassDefFoundError: org/apache/commons/pool/impl/GenericObjectPool异常的解决办法
- getHeader("x-forwarded-for")获取真实IP
- apache产品插件下载官网
- 对ftp和apache代理的初步认识
- php Apache 安装
- php Apache 虚拟主机配置
- Apache和PHP环境搭建
- Apache安装配置
- Windows10 Apache2.4 PHP7 MySQL 5.7安装教程
- Apache CXFjar包目录(转)
- Apache Ranger0.6版本源码编译
- 解决apache启动错误:Could not reliably determine the server's fully qualified domain name