From patchwork Wed Jun 11 10:33:12 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Siddhesh Poyarekar X-Patchwork-Id: 1431 Received: (qmail 17389 invoked by alias); 11 Jun 2014 10:31:53 -0000 Mailing-List: contact libc-alpha-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: libc-alpha-owner@sourceware.org Delivered-To: mailing list libc-alpha@sourceware.org Received: (qmail 17379 invoked by uid 89); 11 Jun 2014 10:31:52 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-3.9 required=5.0 tests=AWL, BAYES_00, RP_MATCHES_RCVD, SPF_HELO_PASS, SPF_PASS autolearn=ham version=3.3.2 X-HELO: mx1.redhat.com Date: Wed, 11 Jun 2014 16:03:12 +0530 From: Siddhesh Poyarekar To: Pedro Alves Cc: "Joseph S. Myers" , libc-alpha@sourceware.org Subject: Re: Patch not showing in patchwork Message-ID: <20140611103312.GE24193@spoyarek.pnq.redhat.com> References: <53976045.8050602@redhat.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <53976045.8050602@redhat.com> User-Agent: Mutt/1.5.22.1-rc1 (2013-10-16) On Tue, Jun 10, 2014 at 08:45:09PM +0100, Pedro Alves wrote: > On 06/10/2014 05:39 PM, Joseph S. Myers wrote: > > I noticed that my patch > > does not > > appear in the list of patches in patchwork (it appears not to have a patch > > entry in patchwork at all, rather than having been mistakenly marked as > > committed). Is there some way to tell why it didn't get recognised by > > patchwork? > > > > I'll guess due to: > > Content-Type: text/plain; charset="X-UNKNOWN" This was the problem. I have pushed the patch manually: http://patchwork.sourceware.org/patch/1430/ I have also applied the following patch to patchwork to allow it to fall back on alternate encodings if the charset is unknown. I'll post it to the patchwork mailing list. Siddhesh --- a/apps/patchwork/bin/parsemail.py 2014-06-11 15:53:12.685666812 +0530 +++ b/apps/patchwork/bin/parsemail.py 2014-06-11 15:53:03.991667186 +0530 @@ -147,6 +147,13 @@ return match.group(1) return None +def try_decode(payload, charset): + try: + payload = unicode(payload, charset) + except UnicodeDecodeError: + return None + return payload + def find_content(project, mail): patchbuf = None commentbuf = '' @@ -157,15 +164,27 @@ continue payload = part.get_payload(decode=True) - charset = part.get_content_charset() subtype = part.get_content_subtype() - # if we don't have a charset, assume utf-8 - if charset is None: - charset = 'utf-8' - if not isinstance(payload, unicode): - payload = unicode(payload, charset) + charset = part.get_content_charset() + + # If there is no charset or if it is unknown, then try some common + # charsets before we fail. + if charset is None or charset == 'x-unknown': + try_charsets = ['utf-8', 'windows-1252', 'ascii', 'iso-8859'] + else: + try_charsets = [charset] + + for cset in try_charsets: + decoded_payload = try_decode(payload, cset) + if decoded_payload is not None: + break + payload = decoded_payload + + # Could not find a valid decoded payload. Fail. + if payload is None: + return (None, None) if subtype in ['x-patch', 'x-diff']: patchbuf = payload