close Warning: Can't synchronize with repository "(default)" (/work/source/svn/csep does not appear to be a Subversion repository.). Look in the Trac log for more information.

Opened 6 years ago

Closed 6 years ago

#277 closed defect (fixed)

Failure to dowload one of existing CMT *ndk final calalogs results in earlier date used for preliminary catalog

Reported by: liukis Owned by: liukis
Priority: major Milestone: CSEP Version 12.4.0
Component: NaturalLaboratory Version: 12.1.0
Keywords: Cc:

Description

While testing archival of existing CMT catalog for SW and NW Pacific testing regions on csep-cert an exception was risen when raw CMT catalog downloaded at exactly the same runtime (resulting in exactly the same SVN tag for raw catalog) was different for NW vs SW Pacific testing region. Due to the failure to download one of existing *.ndk files:

[csep@csep-cert NWPacific]$ cat /home/csep/operations/dispatcher/runs/NWPacific/csep/2012_02/20120209000503/pid_8635/dec10.ndk 
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>502 Proxy Error</title>
</head><body>
<h1>Proxy Error</h1>
<p>The proxy server received an invalid
response from an upstream server.<br />
The proxy server could not handle the request <em><a href="/~gcmt/projects/CMT/catalog/NEW_MONTHLY/2010/dec10.ndk">GET&nbsp;/~gcmt/projects/CMT/catalog/NEW_MONTHLY/2010/dec10.ndk</a></em>.<p>
Reason: <strong>Error reading from remote server</strong></p></p>
<hr>
<address>Apache/2.2.3 (CentOS) Server at www.ldeo.columbia.edu Port 80</address>
</body></html>
[csep@csep-cert NWPacific]$

triggered earlier date to be used for preliminary catalog (/home/csep/operations/dispatcher/logs/WesternPacific/2012_2/nw_daily_2012-2-9-000502 log file):

Thu, 2012-02-09 00:05:43 CSEP INFO     Downloaded 'dec10.ndk' file content has unexpected type: got 'text/html; charset=iso-8859-1',    expected 'text/plain'
Thu, 2012-02-09 00:05:43 CSEP INFO     Downloading 2010 from http://www.ldeo.columbia.edu/~gcmt/projects/CMT/catalog/NEW_MONTHLY
Thu, 2012-02-09 00:05:44 CSEP INFO     Downloaded '2010' file content has unexpected type: got 'text/html;charset=ISO-8859-1',    expected 'text/plain'
Thu, 2012-02-09 00:05:44 CMTDataSource INFO     Quick CMT catalog start date is 2010-12-01
Thu, 2012-02-09 00:05:44 CSEP INFO     Downloading qcmt.ndk from http://www.ldeo.columbia.edu/~gcmt/projects/CMT/catalog/NEW_QUICK
Thu, 2012-02-09 00:05:45 CMTDataSource INFO     Appending quick events beginning with 2010/12/01

Change History (3)

comment:1 Changed 6 years ago by liukis

  • Owner set to liukis
  • Status changed from new to accepted

comment:2 Changed 6 years ago by liukis

Changed data retrieval to use urllib2 instead of urllib module for easier error code parsing

comment:3 Changed 6 years ago by liukis

  • Resolution set to fixed
  • Status changed from accepted to closed
Note: See TracTickets for help on using tickets.