Skip to content

When reusing an handler, urllib(2)'s digest authentication fails after multiple regative replies #53907

Open

Description

@LuciStanescu
BPO 9698
Nosy @orsenthil, @spaceone
Files
  • test_urllib2.py
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = None
    created_at = <Date 2010-08-27.06:59:26.263>
    labels = ['type-bug', 'library']
    title = "When reusing an handler, urllib(2)'s digest authentication fails after multiple regative replies"
    updated_at = <Date 2019-03-15.22:16:19.917>
    user = 'https://bugs.python.org/LuciStanescu'

    bugs.python.org fields:

    activity = <Date 2019-03-15.22:16:19.917>
    actor = 'BreamoreBoy'
    assignee = 'none'
    closed = False
    closed_date = None
    closer = None
    components = ['Library (Lib)']
    creation = <Date 2010-08-27.06:59:26.263>
    creator = 'Luci.Stanescu'
    dependencies = []
    files = ['18656']
    hgrepos = []
    issue_num = 9698
    keywords = []
    message_count = 3.0
    messages = ['115054', '220849', '235846']
    nosy_count = 5.0
    nosy_names = ['orsenthil', 'kiilerix', 'Luci.Stanescu', 'Erick.Jones', 'spaceone']
    pr_nums = []
    priority = 'normal'
    resolution = None
    stage = None
    status = 'open'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue9698'
    versions = ['Python 2.6', 'Python 3.1', 'Python 2.7', 'Python 3.2']

    Linked PRs

    Activity

    1. LuciStanescu commented on Aug 27, 2010

      LuciStanescumannequin
      MannequinAuthor

      Hi,

      The HTTPDigestAuthHandler's code looks like this:

          def http_error_401(self, req, fp, code, msg, headers):
              host = urlparse(req.full_url)[1]
              retry = self.http_error_auth_reqed('www-authenticate',
                                                 host, req, headers)
              self.reset_retry_count()
              return retry

      After successful authentication, the HTTP server might still return an error code, such as 404 or 304. In that case, self.http_error_auth_reqed raises the appropriate HTTPError and self.reset_retry_count is not called. I think that the code should be something along the lines of:

      try:
      retry = self.http_error_auth_reqed('www-authenticate', host, req, headers)
      except HTTPError, e:
      if e.code != 401:
      self.reset_retry_counter()
      raise
      else:
      self.reset_retry_counter()
      return retry

      Ways to reproduce the problem: try to access a resource for which an HTTP server requires authentication but for which after successful authentication returns a negative reply. I've attached an example script to demonstrate it (for python 2.X; bug also resent in 3.X, just replace import urllib2 with from urllib import request as urllib2 ;-) ).

      The same problem applies to ProxyDigestAuthHandler.

    2. added
      stdlibStandard Library Python modules in the Lib/ directory
      type-bugAn unexpected behavior, bug, or error
      on Aug 27, 2010
    3. BreamoreBoy commented on Jun 17, 2014

      BreamoreBoymannequin
      Mannequin

      Could we have a response to this please as a way to reproduce the problem is given in the attached patch and a suggested solution is inline.

    4. ErickJones commented on Feb 12, 2015

      ErickJonesmannequin
      Mannequin

      This ended up biting me also. I had a list of URLs to fetch with authentication. One of the URLs was bad (returning 401 even with authentication), and that was causing all of the subsequent URLs to fail as well since the reset count wasn't getting reset.

      I also don't like that the retry count is stored in the handler -- that's mutable global state, which wreaks havoc if I use this with Eventlet coroutines for concurrent page fetches. (If I just add the authentication headers myself, then urllib2 works just fine under Eventlet.)

      Couldn't the retry count be stored in the request object itself?

      And why do we even need a retry "count"? If it fails without authentication, then try it with authentication. If it fails again, just return to the application. It makes no sense to retry four more times.

    5. transferred this issue fromon Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    No one assigned

      Labels

      stdlibStandard Library Python modules in the Lib/ directorytype-bugAn unexpected behavior, bug, or error

      Projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions