AWS Glue Enhances Spark Engines and Supports the Ray Framework


Share post:

AWS Glue, a serverless data integration service provided by Amazon Web Services, showcases Python and Apache Spark capabilities in a version 4.0 release introduced this week.

The update includes engines for Apache Spark 3.3.0 and Python 3.10. Both engines have performance improvements and bug fixes, with Spark providing features like improved error messages and row-level runtime filtering. The Cloud Shuffle Service for Spark, the Ray compute framework, and Adaptive Query Execution are supported by new engine plugins in Glue 4.0. Also included is support for the Python-based Pandas data analysis and manipulation tool. Delta Lake, Apache Iceberg, and Apache Hudi all now have new data format support. The Parquet vectorized reader, with support for additional encodings and data types, is also part of Glue 4.0.

Read More: AWS Glue upgrades Spark engines, backs Ray framework

TalkDev Bureau
TalkDev Bureau
The TalkDev Bureau has five well-trained writers and journalists, well versed in B2B enterprise technology industry, and constantly in touch with industry leaders for the latest trends, opinions, and other inputs- to bring you the best and latest in the domain.


Please enter your comment!
Please enter your name here


Related articles

Synopsys and Arm Partner to Rapid Bring-Up of Next-Generation Mobile SoC Designs on Advanced Nodes

Synopsys and Arm strengthen their partnership by declaring the Arm® Total Compute Solutions 2023 (TCS23) platform at Computex....

N-iX Collaborates with Mendix, a low-code Platform Provider

N-iX, a software and engineering solutions company, declared a partnership with Mendix. The collaboration helps clients speed up...

Onymos – Feature-as-a-Service: Making Your APP Development Journey Easy

Onymos is a popular choice among developers for its easy of use, scalability, and range of features. Onymos is...

Comcast Launches Internet Service Based on Xfinity’s 10G Network to Arcadia

Comcast is accelerating the nation's fastest and largest multi-gig rollout and today announced it is expanding Xfinity's 10G network to...