Details about using MaxCompute:
Apsara Clouder Big Data Special Skills Certification: Use MaxCompute to build a social friend recommendation system
This certification can help students understand how to use Alibaba Cloud big data computing services to quickly build an enterprise-level social friend recommendation system, and master the ability to use cloud-related big data services for development and testing.
Big Data Computing Service (MaxCompute, formerly known as ODPS) is a fast and fully managed GB/TB/PB-level data warehouse solution. MaxCompute provides you with a complete data import solution and a variety of classic distributed computing models, which can solve massive data computing problems more quickly, effectively reduce enterprise costs, and ensure data security.
At the same time, DataWorks and MaxCompute are closely related. DataWorks provides MaxCompute with one-stop data synchronization, task development, data workflow development, data management, and data operation and maintenance functions. For details, please refer to DataWorks (formerly Big Data Development Kit).
MaxCompute mainly serves the storage and calculation of batch structured data, and can provide solutions for massive data warehouses and analysis and modeling services for big data. With the continuous enrichment and improvement of social data collection methods, more and more industry data have been accumulated. The data scale has grown to the massive data (hundred GB, TB or even PB) level that the traditional software industry cannot carry.
In the scenario of analyzing massive data, due to the limitation of the processing capacity of a single server, data analysts usually adopt a distributed computing model. However, the distributed computing model places higher requirements on data analysts and is not easy to maintain. Using the distributed model, data analysts not only need to understand business requirements, but also need to be familiar with the underlying computing model. The purpose of MaxCompute is to provide you with a convenient means of analyzing and processing massive amounts of data. You can achieve the purpose of analyzing big data without worrying about the details of distributed computing.
MaxCompute has been widely used within Alibaba Group, such as data warehouse and BI analysis of large Internet companies, log analysis of websites, transaction analysis of e-commerce websites, user characteristics and interest mining, etc. Product advantages
Large-scale computing and storage
MaxCompute is suitable for storage and computing requirements with a scale of 100GB or more, up to the EB level.
Multiple calculation models
MaxCompute supports calculation types such as SQL, MapReduce, Graph, and MPI iteration algorithms.
Strong data security
MaxCompute has stably supported all of Ali's offline analysis services for more than 7 years, providing multi-layer sandbox protection and monitoring.
Compared with self-built private clouds, MaxCompute's computing storage is more efficient and can reduce procurement costs by 20%-30%.
Support for batch and historical data channels TUNNEL is a data transmission service provided by MaxCompute for you, providing high-concurrency offline data upload and download services. Support daily TB/PB level data import and export, especially suitable for batch import of full data or historical data. Tunnel provides a Java programming interface for you to use, and in the MaxCompute client tool, there are corresponding commands to realize the intercommunication between local files and service data.
Real-time and incremental data channels For real-time data upload scenarios, MaxCompute provides a DataHub service with low latency and easy-to-use, which is especially suitable for the import of incremental data. DataHub also supports a variety of data transmission plug-ins, such as Logstash, Flume, Fluentd, Sqoop, etc. It also supports one-click delivery of log data in Log Service to MaxCompute, and then uses DataWorks for log analysis and mining.
More quality courses:
7 days to play cloud server
ApsaraDB for Redis edition tutorial
Getting started with cloud storage object storage OSS
Alibaba Cloud CDN usage tutorial
Introduction to Load Balancing and Product Usage Guide
Alibaba Cloud University official website (Alibaba Cloud University-official website, innovative talent workshop under the cloud ecology)