[jira] [Updated] (HADOOP-14547) [WASB] the configured retry policy is not used for all storage operations.

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[jira] [Updated] (HADOOP-14547) [WASB] the configured retry policy is not used for all storage operations.

JIRA jira@apache.org

     [ https://issues.apache.org/jira/browse/HADOOP-14547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Thomas updated HADOOP-14547:
----------------------------
    Attachment: HADOOP-14547.001.patch

Attaching patch HADOOP-14547.001.patch.

The issue is fixed by reading the WASB configuration settings before creating and initializing the storage components (CloudBlobClient, CloudBlobContainer, etc).  The retry policy and session timeout are now stored by the StorageInterface implementations and used to properly initialize the storage objects.

Unfortunately, the retry policy cannot easily be validated with unit tests due to functionality that is not exposed by the Azure Storage SDK.  I have carefully validated this change in the debugger to ensure that the user configured retry policy is used.

> [WASB] the configured retry policy is not used for all storage operations.
> --------------------------------------------------------------------------
>
>                 Key: HADOOP-14547
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14547
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs/azure
>    Affects Versions: 2.9.0, 3.0.0-alpha4
>            Reporter: Thomas
>            Assignee: Thomas
>             Fix For: 2.9.0, 3.0.0-alpha4
>
>         Attachments: HADOOP-14547.001.patch
>
>
> There are a few places where the WASB retry policy was not used, and instead the Azure Storage SDK default retry policy was used.  These places include some calls to blob.exist(), container.exists(), blob.delete(), and operations with secure mode enabled (fs.azure.secure.mode = true).  
> You can set a break point on com.microsoft.azure.storage.core.ExecutionEngine.executeWithRetry in the Azure Storage SDK and see that the WASB configured retry policy (DEFAULT_MAX_RETRY_ATTEMPTS, etc.) is not used.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Loading...