In addition to writing shuffle files, executors also cache data either on disk or in memory. When an executor is removed, however, all cached data will no longer be accessible. To mitigate this, by default executors containing cached data are never removed. You can configure this behavior with spark.dynamicAllocation.cachedExecutorIdleTimeout. In future releases, the cached data may be preserved through an off-heap storage similar in spirit to how shuffle files are preserved through the external shuffle service. From official spark documentation
Hello Jay, if u hve material (like ppts PDFs videos or any other)regarding the allocation of resources in the cloud for media streaming application do send it to me on dzambare@rediffmail.com
Can anyone answer to that last question.. That is damn good question...?
In addition to writing shuffle files, executors also cache data either on disk or in memory. When an executor is removed, however, all cached data will no longer be accessible. To mitigate this, by default executors containing cached data are never removed. You can configure this behavior with spark.dynamicAllocation.cachedExecutorIdleTimeout. In future releases, the cached data may be preserved through an off-heap storage similar in spirit to how shuffle files are preserved through the external shuffle service.
From official spark documentation
A
Hard to understand the way he speak. Its like he’s mumbling or something
English à la française
Hello Jay, if u hve material (like ppts PDFs videos or any other)regarding the allocation of resources in the cloud for media streaming application do send it to me on dzambare@rediffmail.com