Greenplum中的资源管理策略（25页）.pdf

1、Greenplum中的资源管理策略王尧,主任工程师Greenplum，VMware2021.11Index Greenplum简介 基于角色的资源管理 Resource Queue Resource Group 转换策略需要注意的问题 FAQ 参考文档Greenplum简介Greenplum 是全球领先的大数据分析引擎，专为分析、机器学习和AI而打造 线性扩展能力：大规模并行处理架构，PB级存储、处理和实时分析能力 支持多种数据源和数据格式：括 Hadoop、HIVE、HBase、S3、Gemfire、各种数据库和文件等 多态数据存储：支持堆表，以及面向行存或列存的追加表 集成数据库分析处理数

2、据科学：内建可扩展数据库分析开源库Apache MADlib Greenplum提供的查询优化器是业界优秀的开源的基于代价的查询优化器Pivotal公司产品，现在属于VMware公司基于角色的资源管理 基于角色的资源管理 CPU 内存 并发数 目前支持两种策略 Resource Queue Resource Group RQ和RG的选择（差异见下页）Resource Queue和Resource Groups的差异一览MetricResource QueuesResource GroupsConcurrencyManaged at the query levelManaged at the t

3、ransaction levelCPUSpecify query prioritySpecify percentage of CPU resources;uses Linux Control GroupsMemoryManaged at the query and operator level;users can over-subscribeManaged at the transaction level,with enhanced allocation and tracking;users cannot over-subscribeMemory IsolationNoneMemory is

4、isolated between resource groups and between transactions within the same resource groupUsersLimits are applied only to non-admin usersLimits are applied to SUPERUSER and non-admin users alikeQueueingQueue only when no slot availableQueue when no slot is available or not enough available memoryQuery

5、 FailureQuery may fail immediately if not enough memoryQuery may fail after reaching transaction fixed memory limit when no shared resource group memory exists and the transaction requests more memoryLimit BypassLimits are not enforced for SUPERUSER roles and certain operators and functionsLimits ar

6、e not enforced on SET,RESET,and SHOW commandsExternal ComponentsNoneManage PL/Container CPU and memory resourcesResource Queue:范例 CREATE RESOURCE QUEUE myqueue WITH(ACTIVE_STATEMENTS=20,MEMORY_LIMIT=2000MB);CREATE ROLE myname WITH LOGIN RESOURCE QUEUE myqueue;SET ROLE myname;Resource Queue:范例SELECT*