What Is Apache Spark
2017-09-27 10:11
260 查看
What Is Apache Spark?
Apache Spark is a cluster computing platform designed to be fast and general-purpose. One of the main features Spark offers for speed is the ability to run computations in memory.Spark is designed to be highly accessible, offering simple APIs in Python, Java, Scala, and SQL, and rich built-in libraries. It also integrates closely with other Big Data tools. In particular, Spark can run in Hadoop clusters and access any Hadoop data source, including Cassandra.
Spark Core
Spark Core is home to the API that defines resilient distributed datasets (RDDs), which are Spark’s main programming abstraction. RDDs represent a collection of items distributed across many compute nodes that can be manipulated in parallel.Spark SQL
Spark SQL is Spark’s package for working with structured data. It allows querying data via SQL as well as the Apache Hive variant of SQL — called the Hive Query Language(HQL) .Spark Streaming
Spark Streaming is a Spark component that enables processing of live streams of data.MLlib
Spark comes with a library containing common machine learning (ML) functionality, called MLlib. MLlib provides multiple types of machine learning algorithms.GraphX
GraphX is a library for manipulating graphs (e.g., a social network’s friend graph) and performing graph-parallel computations.Cluster Managers
Under the hood, Spark is designed to efficiently scale up from one to many thousands of compute nodes. To achieve this while maximizing flexibility, Spark can run over a variety of cluster managers, including Hadoop YARN, Apache Mesos, and a simple cluster manager included in Spark itself called the Standalone Scheduler.相关文章推荐
- scala学习-Description Resource Path Location Type value toDF is not a member of org.apache.spark.rdd.R
- 解决value toDF is not a member of org.apache.spark.rdd.RDD[People]
- 1.1 Introduction中 Apache Kafka™ is a distributed streaming platform. What exactly does that mean?(官网剖析)(博主推荐)
- What’s new for Spark SQL in Apache Spark 1.3(中英双语)
- What is the difference Apache (Http Server) and Tomcat (Servlet Container)
- value toDF is not a member of org.apache.spark.rdd.RDD[People]
- error: object kafka is not a member of package org.apache.spark.streaming
- What Is Apache Hadoop?
- 拼写错误:value countBykey is not a member of org.apache.spark.rdd.RDD[(String, Int)]
- value toDF is not a member of org.apache.spark.rdd.RDD
- Spark-000讲:What is Spark ?
- Is Apache Spark the Next Big Thing in Big Data?
- value toDF is not a member of org.apache.spark.rdd.RDD[People]
- Why Apache Spark is a Crossover Hit for Data Scientists [FWD]
- log4j:ERROR A "org.jboss.logging.appender.FileAppender" object is not assignable to a "org.apache.log4j.Appender" variable.
- 1.1. What Is A Kernel Module?
- What do you think is the most important skill every programmer should posses?(转)
- what is pdf
- MyBatis批量更新报错org.mybatis.spring.MyBatisSystemException: nested exception is org.apache.ibatis.bindin
- what is Activity in Android