DATA ENGINEERING
Spark SQL
Spark submit options and tuning notes covering executors, memory, parallelism, and resource allocation on YARN.
Tag / 3 articles
#optimization
Blog posts tagged with optimization
DATA ENGINEERING
Spark Optimization
Data locality is how close data is to the code processing it. There are several levels of locality based on the data’s current location. In order from closest to farthest:
MACHINE LEARNING
Gradient Descent
Vanilla gradient descent, aka batch gradient descent, computes the gradient of the cost function w.r.t. to the parameters θ