SportsSpiderlog20190622.txt 101 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432433434435436437438439440441442443444445446447448449450451452453454455456457458459460461462463464465466467468469470471472473474475476477478479480481482483484485486487488489490491492493494495496497498499500501502503504505506507508509510511512513514515516517518519520521522523524525526527528529530531532533534535536537538539540541542543544545546547548549550551552553554555556557558559560561562563564565566567568569570571572573574575576577578579580581582583584585586587588589590591592593594595596597598599600601602603604605606607608609610611612613614615616617618619620621622623624625626627628629630631632633634635636637638639640641642643644645646647648649650651652653654655656657658659660661662663664665666667668669670671672673674675676677678679680681682683684685686687688689690691692693694695696697698699700701702703704705706707708709710711712713714715716717718719720721722723724725726727728729730731732733734735736737738739740741742743744745746747748749750751752753754755756757758759760761762763764765766767768769770771772773774775776777778779780781782783784785786787788789790791792793794795796797798799800801802803804805806807808809810811812813814815816817818819820821822823824825826827828829830831832833834835836837838839840841842843844845846847848849850851852853854855856857858859860861862863864865866867868869870871872873874875876877878879880881882883884885886887888889890891892893894895896897898899900901902903904905906907908909910911912913914915916917918919920921922923924925926927928929930931932933934935936937938939940941942943944945946947948949950951952953954955956957958959960961962963964965966967968969970971972973974975976977978979980981982983984985986987988989990991992993994995996997998999100010011002100310041005100610071008100910101011101210131014101510161017101810191020102110221023102410251026102710281029103010311032103310341035103610371038103910401041104210431044104510461047104810491050105110521053105410551056105710581059106010611062106310641065106610671068106910701071107210731074107510761077107810791080108110821083108410851086108710881089109010911092109310941095109610971098109911001101110211031104110511061107110811091110111111121113111411151116111711181119112011211122112311241125112611271128112911301131113211331134113511361137113811391140114111421143114411451146114711481149115011511152115311541155115611571158115911601161116211631164116511661167116811691170117111721173117411751176117711781179118011811182118311841185118611871188118911901191119211931194119511961197119811991200120112021203120412051206120712081209121012111212121312141215121612171218121912201221122212231224122512261227122812291230123112321233123412351236123712381239124012411242124312441245124612471248124912501251125212531254125512561257125812591260126112621263126412651266126712681269127012711272127312741275127612771278127912801281128212831284128512861287128812891290129112921293129412951296129712981299130013011302130313041305
  1. 2019-06-22 09:08:13 [scrapy.utils.log] INFO: Scrapy 1.6.0 started (bot: SportsSpider)
  2. 2019-06-22 09:08:13 [scrapy.utils.log] INFO: Versions: lxml 4.3.2.0, libxml2 2.9.9, cssselect 1.0.3, parsel 1.5.1, w3lib 1.20.0, Twisted 19.2.1, Python 3.7.3 (default, Mar 27 2019, 17:13:21) [MSC v.1915 64 bit (AMD64)], pyOpenSSL 19.0.0 (OpenSSL 1.1.1b 26 Feb 2019), cryptography 2.6.1, Platform Windows-10-10.0.17763-SP0
  3. 2019-06-22 09:08:13 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'SportsSpider', 'LOG_FILE': 'SportsSpiderlog20190622.txt', 'LOG_LEVEL': 'INFO', 'NEWSPIDER_MODULE': 'collectSports.spiders', 'SPIDER_MODULES': ['collectSports.spiders'], 'USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 UBrowser/6.2.4098.3 Safari/537.36'}
  4. 2019-06-22 09:08:13 [scrapy.extensions.telnet] INFO: Telnet Password: 66adf74d240537aa
  5. 2019-06-22 09:08:13 [scrapy.middleware] INFO: Enabled extensions:
  6. ['scrapy.extensions.corestats.CoreStats',
  7. 'scrapy.extensions.telnet.TelnetConsole',
  8. 'scrapy.extensions.logstats.LogStats']
  9. 2019-06-22 09:08:13 [scrapy.middleware] INFO: Enabled downloader middlewares:
  10. ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
  11. 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
  12. 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
  13. 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
  14. 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
  15. 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
  16. 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
  17. 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
  18. 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
  19. 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
  20. 'scrapy.downloadermiddlewares.stats.DownloaderStats']
  21. 2019-06-22 09:08:13 [scrapy.middleware] INFO: Enabled spider middlewares:
  22. ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
  23. 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
  24. 'scrapy.spidermiddlewares.referer.RefererMiddleware',
  25. 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
  26. 'scrapy.spidermiddlewares.depth.DepthMiddleware']
  27. 2019-06-22 09:08:14 [scrapy.middleware] INFO: Enabled item pipelines:
  28. ['collectSports.pipelines.sportslst.SportslstPipeline']
  29. 2019-06-22 09:08:14 [scrapy.core.engine] INFO: Spider opened
  30. 2019-06-22 09:08:14 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
  31. 2019-06-22 09:08:14 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
  32. 2019-06-22 09:08:19 [scrapy.core.downloader.tls] WARNING: Ignoring error while verifying certificate from host "199.26.100.178" (exception: ValueError('Invalid DNS-ID.'))
  33. 2019-06-22 09:09:53 [scrapy.core.scraper] ERROR: Spider error processing <GET https://199.26.100.178/app/member/FT_browse/body_var.php?uid=opqygihvm21680472l14201&rtype=r&langx=zh-cn&mtype=4&page_no=1&league_id=&hot_game=&isie11=27N%27> (referer: None)
  34. Traceback (most recent call last):
  35. File "C:\ProgramData\Anaconda3\lib\site-packages\scrapy\utils\defer.py", line 102, in iter_errback
  36. yield next(it)
  37. File "C:\ProgramData\Anaconda3\lib\site-packages\scrapy\spidermiddlewares\offsite.py", line 29, in process_spider_output
  38. for x in result:
  39. File "C:\ProgramData\Anaconda3\lib\site-packages\scrapy\spidermiddlewares\referer.py", line 339, in <genexpr>
  40. return (_set_referer(r) for r in result or ())
  41. File "C:\ProgramData\Anaconda3\lib\site-packages\scrapy\spidermiddlewares\urllength.py", line 37, in <genexpr>
  42. return (r for r in result or () if _filter(r))
  43. File "C:\ProgramData\Anaconda3\lib\site-packages\scrapy\spidermiddlewares\depth.py", line 58, in <genexpr>
  44. return (r for r in result or () if _filter(r))
  45. File "D:\workspace\sports_collect\collectSports\spiders\sportslst.py", line 58, in parse
  46. re = cbk(response.body)
  47. File "D:\workspace\sports_collect\collectSports\mcollect\hg0088\Resolver.py", line 20, in today
  48. re = self.mixbodyv(data, 'today', '/html/head/script[2]/text()')
  49. File "D:\workspace\sports_collect\collectSports\mcollect\hg0088\Resolver.py", line 35, in mixbodyv
  50. text = ls[0]
  51. IndexError: list index out of range
  52. 2019-06-22 09:09:53 [scrapy.extensions.logstats] INFO: Crawled 1 pages (at 1 pages/min), scraped 0 items (at 0 items/min)
  53. 2019-06-22 09:09:53 [scrapy.core.engine] INFO: Closing spider (finished)
  54. 2019-06-22 09:09:53 [scrapy.core.engine] ERROR: Scraper close failure
  55. Traceback (most recent call last):
  56. File "C:\ProgramData\Anaconda3\lib\site-packages\twisted\internet\defer.py", line 654, in _runCallbacks
  57. current.result = callback(current.result, *args, **kw)
  58. File "D:\workspace\sports_collect\collectSports\pipelines\sportslst.py", line 167, in close_spider
  59. self.client.close()
  60. AttributeError: 'SportslstPipeline' object has no attribute 'client'
  61. 2019-06-22 09:09:53 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
  62. {'downloader/request_bytes': 442,
  63. 'downloader/request_count': 1,
  64. 'downloader/request_method_count/GET': 1,
  65. 'downloader/response_bytes': 306,
  66. 'downloader/response_count': 1,
  67. 'downloader/response_status_count/200': 1,
  68. 'finish_reason': 'finished',
  69. 'finish_time': datetime.datetime(2019, 6, 22, 1, 9, 53, 348330),
  70. 'log_count/ERROR': 2,
  71. 'log_count/INFO': 10,
  72. 'log_count/WARNING': 1,
  73. 'response_received_count': 1,
  74. 'scheduler/dequeued': 1,
  75. 'scheduler/dequeued/memory': 1,
  76. 'scheduler/enqueued': 1,
  77. 'scheduler/enqueued/memory': 1,
  78. 'spider_exceptions/IndexError': 1,
  79. 'start_time': datetime.datetime(2019, 6, 22, 1, 8, 14, 84727)}
  80. 2019-06-22 09:09:53 [scrapy.core.engine] INFO: Spider closed (finished)
  81. 2019-06-22 09:10:02 [scrapy.utils.log] INFO: Scrapy 1.6.0 started (bot: SportsSpider)
  82. 2019-06-22 09:10:02 [scrapy.utils.log] INFO: Versions: lxml 4.3.2.0, libxml2 2.9.9, cssselect 1.0.3, parsel 1.5.1, w3lib 1.20.0, Twisted 19.2.1, Python 3.7.3 (default, Mar 27 2019, 17:13:21) [MSC v.1915 64 bit (AMD64)], pyOpenSSL 19.0.0 (OpenSSL 1.1.1b 26 Feb 2019), cryptography 2.6.1, Platform Windows-10-10.0.17763-SP0
  83. 2019-06-22 09:10:02 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'SportsSpider', 'LOG_FILE': 'SportsSpiderlog20190622.txt', 'LOG_LEVEL': 'INFO', 'NEWSPIDER_MODULE': 'collectSports.spiders', 'SPIDER_MODULES': ['collectSports.spiders'], 'USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 UBrowser/6.2.4098.3 Safari/537.36'}
  84. 2019-06-22 09:10:02 [scrapy.extensions.telnet] INFO: Telnet Password: c5dee860fe1a6f70
  85. 2019-06-22 09:10:02 [scrapy.middleware] INFO: Enabled extensions:
  86. ['scrapy.extensions.corestats.CoreStats',
  87. 'scrapy.extensions.telnet.TelnetConsole',
  88. 'scrapy.extensions.logstats.LogStats']
  89. 2019-06-22 09:10:02 [scrapy.middleware] INFO: Enabled downloader middlewares:
  90. ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
  91. 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
  92. 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
  93. 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
  94. 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
  95. 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
  96. 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
  97. 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
  98. 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
  99. 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
  100. 'scrapy.downloadermiddlewares.stats.DownloaderStats']
  101. 2019-06-22 09:10:02 [scrapy.middleware] INFO: Enabled spider middlewares:
  102. ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
  103. 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
  104. 'scrapy.spidermiddlewares.referer.RefererMiddleware',
  105. 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
  106. 'scrapy.spidermiddlewares.depth.DepthMiddleware']
  107. 2019-06-22 09:10:02 [scrapy.middleware] INFO: Enabled item pipelines:
  108. ['collectSports.pipelines.sportslst.SportslstPipeline']
  109. 2019-06-22 09:10:02 [scrapy.core.engine] INFO: Spider opened
  110. 2019-06-22 09:10:02 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
  111. 2019-06-22 09:10:02 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
  112. 2019-06-22 09:10:02 [scrapy.core.downloader.tls] WARNING: Ignoring error while verifying certificate from host "199.26.100.178" (exception: ValueError('Invalid DNS-ID.'))
  113. 2019-06-22 09:10:07 [scrapy.core.scraper] ERROR: Spider error processing <GET https://199.26.100.178/app/member/FT_browse/body_var.php?uid=opqygihvm21680472l14201&rtype=r&langx=zh-cn&mtype=4&page_no=1&league_id=&hot_game=&isie11=27N%27> (referer: None)
  114. Traceback (most recent call last):
  115. File "C:\ProgramData\Anaconda3\lib\site-packages\scrapy\utils\defer.py", line 102, in iter_errback
  116. yield next(it)
  117. File "C:\ProgramData\Anaconda3\lib\site-packages\scrapy\spidermiddlewares\offsite.py", line 29, in process_spider_output
  118. for x in result:
  119. File "C:\ProgramData\Anaconda3\lib\site-packages\scrapy\spidermiddlewares\referer.py", line 339, in <genexpr>
  120. return (_set_referer(r) for r in result or ())
  121. File "C:\ProgramData\Anaconda3\lib\site-packages\scrapy\spidermiddlewares\urllength.py", line 37, in <genexpr>
  122. return (r for r in result or () if _filter(r))
  123. File "C:\ProgramData\Anaconda3\lib\site-packages\scrapy\spidermiddlewares\depth.py", line 58, in <genexpr>
  124. return (r for r in result or () if _filter(r))
  125. File "D:\workspace\sports_collect\collectSports\spiders\sportslst.py", line 58, in parse
  126. re = cbk(response.body)
  127. File "D:\workspace\sports_collect\collectSports\mcollect\hg0088\Resolver.py", line 20, in today
  128. re = self.mixbodyv(data, 'today', '/html/head/script[2]/text()')
  129. File "D:\workspace\sports_collect\collectSports\mcollect\hg0088\Resolver.py", line 35, in mixbodyv
  130. text = ls[0]
  131. IndexError: list index out of range
  132. 2019-06-22 09:10:07 [scrapy.core.engine] INFO: Closing spider (finished)
  133. 2019-06-22 09:10:07 [scrapy.core.engine] ERROR: Scraper close failure
  134. Traceback (most recent call last):
  135. File "C:\ProgramData\Anaconda3\lib\site-packages\twisted\internet\defer.py", line 654, in _runCallbacks
  136. current.result = callback(current.result, *args, **kw)
  137. File "D:\workspace\sports_collect\collectSports\pipelines\sportslst.py", line 167, in close_spider
  138. self.client.close()
  139. AttributeError: 'SportslstPipeline' object has no attribute 'client'
  140. 2019-06-22 09:10:07 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
  141. {'downloader/request_bytes': 442,
  142. 'downloader/request_count': 1,
  143. 'downloader/request_method_count/GET': 1,
  144. 'downloader/response_bytes': 306,
  145. 'downloader/response_count': 1,
  146. 'downloader/response_status_count/200': 1,
  147. 'finish_reason': 'finished',
  148. 'finish_time': datetime.datetime(2019, 6, 22, 1, 10, 7, 465580),
  149. 'log_count/ERROR': 2,
  150. 'log_count/INFO': 9,
  151. 'log_count/WARNING': 1,
  152. 'response_received_count': 1,
  153. 'scheduler/dequeued': 1,
  154. 'scheduler/dequeued/memory': 1,
  155. 'scheduler/enqueued': 1,
  156. 'scheduler/enqueued/memory': 1,
  157. 'spider_exceptions/IndexError': 1,
  158. 'start_time': datetime.datetime(2019, 6, 22, 1, 10, 2, 470597)}
  159. 2019-06-22 09:10:07 [scrapy.core.engine] INFO: Spider closed (finished)
  160. 2019-06-22 09:10:14 [scrapy.utils.log] INFO: Scrapy 1.6.0 started (bot: SportsSpider)
  161. 2019-06-22 09:10:14 [scrapy.utils.log] INFO: Versions: lxml 4.3.2.0, libxml2 2.9.9, cssselect 1.0.3, parsel 1.5.1, w3lib 1.20.0, Twisted 19.2.1, Python 3.7.3 (default, Mar 27 2019, 17:13:21) [MSC v.1915 64 bit (AMD64)], pyOpenSSL 19.0.0 (OpenSSL 1.1.1b 26 Feb 2019), cryptography 2.6.1, Platform Windows-10-10.0.17763-SP0
  162. 2019-06-22 09:10:14 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'SportsSpider', 'LOG_FILE': 'SportsSpiderlog20190622.txt', 'LOG_LEVEL': 'INFO', 'NEWSPIDER_MODULE': 'collectSports.spiders', 'SPIDER_MODULES': ['collectSports.spiders'], 'USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 UBrowser/6.2.4098.3 Safari/537.36'}
  163. 2019-06-22 09:10:14 [scrapy.extensions.telnet] INFO: Telnet Password: fbbf17512304b0d7
  164. 2019-06-22 09:10:14 [scrapy.middleware] INFO: Enabled extensions:
  165. ['scrapy.extensions.corestats.CoreStats',
  166. 'scrapy.extensions.telnet.TelnetConsole',
  167. 'scrapy.extensions.logstats.LogStats']
  168. 2019-06-22 09:10:14 [scrapy.middleware] INFO: Enabled downloader middlewares:
  169. ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
  170. 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
  171. 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
  172. 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
  173. 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
  174. 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
  175. 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
  176. 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
  177. 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
  178. 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
  179. 'scrapy.downloadermiddlewares.stats.DownloaderStats']
  180. 2019-06-22 09:10:14 [scrapy.middleware] INFO: Enabled spider middlewares:
  181. ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
  182. 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
  183. 'scrapy.spidermiddlewares.referer.RefererMiddleware',
  184. 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
  185. 'scrapy.spidermiddlewares.depth.DepthMiddleware']
  186. 2019-06-22 09:10:14 [scrapy.middleware] INFO: Enabled item pipelines:
  187. ['collectSports.pipelines.sportslst.SportslstPipeline']
  188. 2019-06-22 09:10:14 [scrapy.core.engine] INFO: Spider opened
  189. 2019-06-22 09:10:14 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
  190. 2019-06-22 09:10:14 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
  191. 2019-06-22 09:10:14 [scrapy.core.downloader.tls] WARNING: Ignoring error while verifying certificate from host "199.26.100.178" (exception: ValueError('Invalid DNS-ID.'))
  192. 2019-06-22 09:44:50 [scrapy.core.scraper] ERROR: Spider error processing <GET https://199.26.100.178/app/member/FT_browse/body_var.php?uid=opqygihvm21680472l14201&rtype=r&langx=zh-cn&mtype=4&page_no=1&league_id=&hot_game=&isie11=27N%27> (referer: None)
  193. Traceback (most recent call last):
  194. File "C:\ProgramData\Anaconda3\lib\site-packages\scrapy\utils\defer.py", line 102, in iter_errback
  195. yield next(it)
  196. File "C:\ProgramData\Anaconda3\lib\site-packages\scrapy\spidermiddlewares\offsite.py", line 29, in process_spider_output
  197. for x in result:
  198. File "C:\ProgramData\Anaconda3\lib\site-packages\scrapy\spidermiddlewares\referer.py", line 339, in <genexpr>
  199. return (_set_referer(r) for r in result or ())
  200. File "C:\ProgramData\Anaconda3\lib\site-packages\scrapy\spidermiddlewares\urllength.py", line 37, in <genexpr>
  201. return (r for r in result or () if _filter(r))
  202. File "C:\ProgramData\Anaconda3\lib\site-packages\scrapy\spidermiddlewares\depth.py", line 58, in <genexpr>
  203. return (r for r in result or () if _filter(r))
  204. File "D:\workspace\sports_collect\collectSports\spiders\sportslst.py", line 58, in parse
  205. re = cbk(response.body)
  206. File "D:\workspace\sports_collect\collectSports\mcollect\hg0088\Resolver.py", line 20, in today
  207. # // *[ @ id = "title_today"]
  208. File "D:\workspace\sports_collect\collectSports\mcollect\hg0088\Resolver.py", line 35, in mixbodyv
  209. ls = doc.xpath(content_xpath)
  210. IndexError: list index out of range
  211. 2019-06-22 09:44:50 [scrapy.extensions.logstats] INFO: Crawled 1 pages (at 1 pages/min), scraped 0 items (at 0 items/min)
  212. 2019-06-22 09:44:50 [scrapy.core.engine] INFO: Closing spider (finished)
  213. 2019-06-22 09:44:50 [scrapy.core.engine] ERROR: Scraper close failure
  214. Traceback (most recent call last):
  215. File "C:\ProgramData\Anaconda3\lib\site-packages\twisted\internet\defer.py", line 654, in _runCallbacks
  216. current.result = callback(current.result, *args, **kw)
  217. File "D:\workspace\sports_collect\collectSports\pipelines\sportslst.py", line 167, in close_spider
  218. self.client.close()
  219. AttributeError: 'SportslstPipeline' object has no attribute 'client'
  220. 2019-06-22 09:44:50 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
  221. {'downloader/request_bytes': 442,
  222. 'downloader/request_count': 1,
  223. 'downloader/request_method_count/GET': 1,
  224. 'downloader/response_bytes': 306,
  225. 'downloader/response_count': 1,
  226. 'downloader/response_status_count/200': 1,
  227. 'finish_reason': 'finished',
  228. 'finish_time': datetime.datetime(2019, 6, 22, 1, 44, 50, 513810),
  229. 'log_count/ERROR': 2,
  230. 'log_count/INFO': 10,
  231. 'log_count/WARNING': 1,
  232. 'response_received_count': 1,
  233. 'scheduler/dequeued': 1,
  234. 'scheduler/dequeued/memory': 1,
  235. 'scheduler/enqueued': 1,
  236. 'scheduler/enqueued/memory': 1,
  237. 'spider_exceptions/IndexError': 1,
  238. 'start_time': datetime.datetime(2019, 6, 22, 1, 10, 14, 577435)}
  239. 2019-06-22 09:44:50 [scrapy.core.engine] INFO: Spider closed (finished)
  240. 2019-06-22 09:45:00 [scrapy.utils.log] INFO: Scrapy 1.6.0 started (bot: SportsSpider)
  241. 2019-06-22 09:45:00 [scrapy.utils.log] INFO: Versions: lxml 4.3.2.0, libxml2 2.9.9, cssselect 1.0.3, parsel 1.5.1, w3lib 1.20.0, Twisted 19.2.1, Python 3.7.3 (default, Mar 27 2019, 17:13:21) [MSC v.1915 64 bit (AMD64)], pyOpenSSL 19.0.0 (OpenSSL 1.1.1b 26 Feb 2019), cryptography 2.6.1, Platform Windows-10-10.0.17763-SP0
  242. 2019-06-22 09:45:00 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'SportsSpider', 'LOG_FILE': 'SportsSpiderlog20190622.txt', 'LOG_LEVEL': 'INFO', 'NEWSPIDER_MODULE': 'collectSports.spiders', 'SPIDER_MODULES': ['collectSports.spiders'], 'USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 UBrowser/6.2.4098.3 Safari/537.36'}
  243. 2019-06-22 09:45:00 [scrapy.extensions.telnet] INFO: Telnet Password: bf58da38955895de
  244. 2019-06-22 09:45:00 [scrapy.middleware] INFO: Enabled extensions:
  245. ['scrapy.extensions.corestats.CoreStats',
  246. 'scrapy.extensions.telnet.TelnetConsole',
  247. 'scrapy.extensions.logstats.LogStats']
  248. 2019-06-22 09:45:00 [scrapy.middleware] INFO: Enabled downloader middlewares:
  249. ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
  250. 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
  251. 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
  252. 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
  253. 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
  254. 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
  255. 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
  256. 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
  257. 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
  258. 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
  259. 'scrapy.downloadermiddlewares.stats.DownloaderStats']
  260. 2019-06-22 09:45:00 [scrapy.middleware] INFO: Enabled spider middlewares:
  261. ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
  262. 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
  263. 'scrapy.spidermiddlewares.referer.RefererMiddleware',
  264. 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
  265. 'scrapy.spidermiddlewares.depth.DepthMiddleware']
  266. 2019-06-22 09:45:00 [scrapy.middleware] INFO: Enabled item pipelines:
  267. ['collectSports.pipelines.sportslst.SportslstPipeline']
  268. 2019-06-22 09:45:00 [scrapy.core.engine] INFO: Spider opened
  269. 2019-06-22 09:45:00 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
  270. 2019-06-22 09:45:00 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
  271. 2019-06-22 09:45:00 [scrapy.core.downloader.tls] WARNING: Ignoring error while verifying certificate from host "199.26.100.178" (exception: ValueError('Invalid DNS-ID.'))
  272. 2019-06-22 10:08:57 [scrapy.utils.log] INFO: Scrapy 1.6.0 started (bot: SportsSpider)
  273. 2019-06-22 10:08:57 [scrapy.utils.log] INFO: Versions: lxml 4.3.2.0, libxml2 2.9.9, cssselect 1.0.3, parsel 1.5.1, w3lib 1.20.0, Twisted 19.2.1, Python 3.7.3 (default, Mar 27 2019, 17:13:21) [MSC v.1915 64 bit (AMD64)], pyOpenSSL 19.0.0 (OpenSSL 1.1.1b 26 Feb 2019), cryptography 2.6.1, Platform Windows-10-10.0.17763-SP0
  274. 2019-06-22 10:08:57 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'SportsSpider', 'LOG_FILE': 'SportsSpiderlog20190622.txt', 'LOG_LEVEL': 'INFO', 'NEWSPIDER_MODULE': 'collectSports.spiders', 'SPIDER_MODULES': ['collectSports.spiders'], 'USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 UBrowser/6.2.4098.3 Safari/537.36'}
  275. 2019-06-22 10:08:57 [scrapy.extensions.telnet] INFO: Telnet Password: 85f4a4c56defc508
  276. 2019-06-22 10:08:57 [scrapy.middleware] INFO: Enabled extensions:
  277. ['scrapy.extensions.corestats.CoreStats',
  278. 'scrapy.extensions.telnet.TelnetConsole',
  279. 'scrapy.extensions.logstats.LogStats']
  280. 2019-06-22 10:08:57 [scrapy.middleware] INFO: Enabled downloader middlewares:
  281. ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
  282. 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
  283. 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
  284. 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
  285. 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
  286. 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
  287. 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
  288. 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
  289. 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
  290. 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
  291. 'scrapy.downloadermiddlewares.stats.DownloaderStats']
  292. 2019-06-22 10:08:57 [scrapy.middleware] INFO: Enabled spider middlewares:
  293. ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
  294. 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
  295. 'scrapy.spidermiddlewares.referer.RefererMiddleware',
  296. 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
  297. 'scrapy.spidermiddlewares.depth.DepthMiddleware']
  298. 2019-06-22 10:08:57 [scrapy.middleware] INFO: Enabled item pipelines:
  299. ['collectSports.pipelines.sportslst.SportslstPipeline']
  300. 2019-06-22 10:08:57 [scrapy.core.engine] INFO: Spider opened
  301. 2019-06-22 10:08:57 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
  302. 2019-06-22 10:08:57 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
  303. 2019-06-22 10:08:58 [scrapy.core.downloader.tls] WARNING: Ignoring error while verifying certificate from host "205.201.4.177" (exception: ValueError('Invalid DNS-ID.'))
  304. 2019-06-22 10:18:00 [scrapy.utils.log] INFO: Scrapy 1.6.0 started (bot: SportsSpider)
  305. 2019-06-22 10:18:00 [scrapy.utils.log] INFO: Versions: lxml 4.3.2.0, libxml2 2.9.9, cssselect 1.0.3, parsel 1.5.1, w3lib 1.20.0, Twisted 19.2.1, Python 3.7.3 (default, Mar 27 2019, 17:13:21) [MSC v.1915 64 bit (AMD64)], pyOpenSSL 19.0.0 (OpenSSL 1.1.1b 26 Feb 2019), cryptography 2.6.1, Platform Windows-10-10.0.17763-SP0
  306. 2019-06-22 10:18:00 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'SportsSpider', 'LOG_FILE': 'SportsSpiderlog20190622.txt', 'LOG_LEVEL': 'INFO', 'NEWSPIDER_MODULE': 'collectSports.spiders', 'SPIDER_MODULES': ['collectSports.spiders'], 'USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 UBrowser/6.2.4098.3 Safari/537.36'}
  307. 2019-06-22 10:18:00 [scrapy.extensions.telnet] INFO: Telnet Password: b71628a0b76cdb69
  308. 2019-06-22 10:18:00 [scrapy.middleware] INFO: Enabled extensions:
  309. ['scrapy.extensions.corestats.CoreStats',
  310. 'scrapy.extensions.telnet.TelnetConsole',
  311. 'scrapy.extensions.logstats.LogStats']
  312. 2019-06-22 10:18:00 [scrapy.middleware] INFO: Enabled downloader middlewares:
  313. ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
  314. 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
  315. 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
  316. 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
  317. 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
  318. 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
  319. 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
  320. 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
  321. 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
  322. 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
  323. 'scrapy.downloadermiddlewares.stats.DownloaderStats']
  324. 2019-06-22 10:18:00 [scrapy.middleware] INFO: Enabled spider middlewares:
  325. ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
  326. 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
  327. 'scrapy.spidermiddlewares.referer.RefererMiddleware',
  328. 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
  329. 'scrapy.spidermiddlewares.depth.DepthMiddleware']
  330. 2019-06-22 10:18:00 [scrapy.middleware] INFO: Enabled item pipelines:
  331. ['collectSports.pipelines.sportslst.SportslstPipeline']
  332. 2019-06-22 10:18:00 [scrapy.core.engine] INFO: Spider opened
  333. 2019-06-22 10:18:00 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
  334. 2019-06-22 10:18:00 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
  335. 2019-06-22 10:18:00 [scrapy.core.downloader.tls] WARNING: Ignoring error while verifying certificate from host "205.201.4.177" (exception: ValueError('Invalid DNS-ID.'))
  336. 2019-06-22 10:19:24 [scrapy.utils.log] INFO: Scrapy 1.6.0 started (bot: SportsSpider)
  337. 2019-06-22 10:19:24 [scrapy.utils.log] INFO: Versions: lxml 4.3.2.0, libxml2 2.9.9, cssselect 1.0.3, parsel 1.5.1, w3lib 1.20.0, Twisted 19.2.1, Python 3.7.3 (default, Mar 27 2019, 17:13:21) [MSC v.1915 64 bit (AMD64)], pyOpenSSL 19.0.0 (OpenSSL 1.1.1b 26 Feb 2019), cryptography 2.6.1, Platform Windows-10-10.0.17763-SP0
  338. 2019-06-22 10:19:24 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'SportsSpider', 'LOG_FILE': 'SportsSpiderlog20190622.txt', 'LOG_LEVEL': 'INFO', 'NEWSPIDER_MODULE': 'collectSports.spiders', 'SPIDER_MODULES': ['collectSports.spiders'], 'USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 UBrowser/6.2.4098.3 Safari/537.36'}
  339. 2019-06-22 10:19:24 [scrapy.extensions.telnet] INFO: Telnet Password: 781e74930d9ccb9f
  340. 2019-06-22 10:19:24 [scrapy.middleware] INFO: Enabled extensions:
  341. ['scrapy.extensions.corestats.CoreStats',
  342. 'scrapy.extensions.telnet.TelnetConsole',
  343. 'scrapy.extensions.logstats.LogStats']
  344. 2019-06-22 10:19:24 [scrapy.middleware] INFO: Enabled downloader middlewares:
  345. ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
  346. 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
  347. 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
  348. 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
  349. 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
  350. 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
  351. 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
  352. 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
  353. 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
  354. 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
  355. 'scrapy.downloadermiddlewares.stats.DownloaderStats']
  356. 2019-06-22 10:19:24 [scrapy.middleware] INFO: Enabled spider middlewares:
  357. ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
  358. 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
  359. 'scrapy.spidermiddlewares.referer.RefererMiddleware',
  360. 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
  361. 'scrapy.spidermiddlewares.depth.DepthMiddleware']
  362. 2019-06-22 10:19:24 [scrapy.middleware] INFO: Enabled item pipelines:
  363. ['collectSports.pipelines.sportslst.SportslstPipeline']
  364. 2019-06-22 10:19:24 [scrapy.core.engine] INFO: Spider opened
  365. 2019-06-22 10:19:24 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
  366. 2019-06-22 10:19:24 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
  367. 2019-06-22 10:19:25 [scrapy.core.downloader.tls] WARNING: Ignoring error while verifying certificate from host "205.201.4.177" (exception: ValueError('Invalid DNS-ID.'))
  368. 2019-06-22 10:20:49 [scrapy.utils.log] INFO: Scrapy 1.6.0 started (bot: SportsSpider)
  369. 2019-06-22 10:20:49 [scrapy.utils.log] INFO: Versions: lxml 4.3.2.0, libxml2 2.9.9, cssselect 1.0.3, parsel 1.5.1, w3lib 1.20.0, Twisted 19.2.1, Python 3.7.3 (default, Mar 27 2019, 17:13:21) [MSC v.1915 64 bit (AMD64)], pyOpenSSL 19.0.0 (OpenSSL 1.1.1b 26 Feb 2019), cryptography 2.6.1, Platform Windows-10-10.0.17763-SP0
  370. 2019-06-22 10:20:49 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'SportsSpider', 'LOG_FILE': 'SportsSpiderlog20190622.txt', 'LOG_LEVEL': 'INFO', 'NEWSPIDER_MODULE': 'collectSports.spiders', 'SPIDER_MODULES': ['collectSports.spiders'], 'USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 UBrowser/6.2.4098.3 Safari/537.36'}
  371. 2019-06-22 10:20:49 [scrapy.extensions.telnet] INFO: Telnet Password: dff90327902ecc8a
  372. 2019-06-22 10:20:49 [scrapy.middleware] INFO: Enabled extensions:
  373. ['scrapy.extensions.corestats.CoreStats',
  374. 'scrapy.extensions.telnet.TelnetConsole',
  375. 'scrapy.extensions.logstats.LogStats']
  376. 2019-06-22 10:20:49 [scrapy.middleware] INFO: Enabled downloader middlewares:
  377. ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
  378. 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
  379. 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
  380. 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
  381. 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
  382. 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
  383. 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
  384. 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
  385. 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
  386. 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
  387. 'scrapy.downloadermiddlewares.stats.DownloaderStats']
  388. 2019-06-22 10:20:49 [scrapy.middleware] INFO: Enabled spider middlewares:
  389. ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
  390. 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
  391. 'scrapy.spidermiddlewares.referer.RefererMiddleware',
  392. 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
  393. 'scrapy.spidermiddlewares.depth.DepthMiddleware']
  394. 2019-06-22 10:20:49 [scrapy.middleware] INFO: Enabled item pipelines:
  395. ['collectSports.pipelines.sportslst.SportslstPipeline']
  396. 2019-06-22 10:20:49 [scrapy.core.engine] INFO: Spider opened
  397. 2019-06-22 10:20:49 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
  398. 2019-06-22 10:20:49 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
  399. 2019-06-22 10:20:50 [scrapy.core.downloader.tls] WARNING: Ignoring error while verifying certificate from host "205.201.4.177" (exception: ValueError('Invalid DNS-ID.'))
  400. 2019-06-22 10:21:29 [scrapy.utils.log] INFO: Scrapy 1.6.0 started (bot: SportsSpider)
  401. 2019-06-22 10:21:29 [scrapy.utils.log] INFO: Versions: lxml 4.3.2.0, libxml2 2.9.9, cssselect 1.0.3, parsel 1.5.1, w3lib 1.20.0, Twisted 19.2.1, Python 3.7.3 (default, Mar 27 2019, 17:13:21) [MSC v.1915 64 bit (AMD64)], pyOpenSSL 19.0.0 (OpenSSL 1.1.1b 26 Feb 2019), cryptography 2.6.1, Platform Windows-10-10.0.17763-SP0
  402. 2019-06-22 10:21:29 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'SportsSpider', 'LOG_FILE': 'SportsSpiderlog20190622.txt', 'LOG_LEVEL': 'INFO', 'NEWSPIDER_MODULE': 'collectSports.spiders', 'SPIDER_MODULES': ['collectSports.spiders'], 'USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 UBrowser/6.2.4098.3 Safari/537.36'}
  403. 2019-06-22 10:21:29 [scrapy.extensions.telnet] INFO: Telnet Password: 22e1efea74b94993
  404. 2019-06-22 10:21:29 [scrapy.middleware] INFO: Enabled extensions:
  405. ['scrapy.extensions.corestats.CoreStats',
  406. 'scrapy.extensions.telnet.TelnetConsole',
  407. 'scrapy.extensions.logstats.LogStats']
  408. 2019-06-22 10:21:29 [scrapy.middleware] INFO: Enabled downloader middlewares:
  409. ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
  410. 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
  411. 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
  412. 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
  413. 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
  414. 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
  415. 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
  416. 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
  417. 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
  418. 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
  419. 'scrapy.downloadermiddlewares.stats.DownloaderStats']
  420. 2019-06-22 10:21:29 [scrapy.middleware] INFO: Enabled spider middlewares:
  421. ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
  422. 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
  423. 'scrapy.spidermiddlewares.referer.RefererMiddleware',
  424. 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
  425. 'scrapy.spidermiddlewares.depth.DepthMiddleware']
  426. 2019-06-22 10:21:29 [scrapy.middleware] INFO: Enabled item pipelines:
  427. ['collectSports.pipelines.sportslst.SportslstPipeline']
  428. 2019-06-22 10:21:29 [scrapy.core.engine] INFO: Spider opened
  429. 2019-06-22 10:21:29 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
  430. 2019-06-22 10:21:29 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
  431. 2019-06-22 10:21:30 [scrapy.core.downloader.tls] WARNING: Ignoring error while verifying certificate from host "205.201.4.177" (exception: ValueError('Invalid DNS-ID.'))
  432. 2019-06-22 10:28:43 [scrapy.utils.log] INFO: Scrapy 1.6.0 started (bot: SportsSpider)
  433. 2019-06-22 10:28:43 [scrapy.utils.log] INFO: Versions: lxml 4.3.2.0, libxml2 2.9.9, cssselect 1.0.3, parsel 1.5.1, w3lib 1.20.0, Twisted 19.2.1, Python 3.7.3 (default, Mar 27 2019, 17:13:21) [MSC v.1915 64 bit (AMD64)], pyOpenSSL 19.0.0 (OpenSSL 1.1.1b 26 Feb 2019), cryptography 2.6.1, Platform Windows-10-10.0.17763-SP0
  434. 2019-06-22 10:28:43 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'SportsSpider', 'LOG_FILE': 'SportsSpiderlog20190622.txt', 'LOG_LEVEL': 'INFO', 'NEWSPIDER_MODULE': 'collectSports.spiders', 'SPIDER_MODULES': ['collectSports.spiders'], 'USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 UBrowser/6.2.4098.3 Safari/537.36'}
  435. 2019-06-22 10:28:43 [scrapy.extensions.telnet] INFO: Telnet Password: 5335f7f9ba6ceccb
  436. 2019-06-22 10:28:43 [scrapy.middleware] INFO: Enabled extensions:
  437. ['scrapy.extensions.corestats.CoreStats',
  438. 'scrapy.extensions.telnet.TelnetConsole',
  439. 'scrapy.extensions.logstats.LogStats']
  440. 2019-06-22 10:28:43 [scrapy.middleware] INFO: Enabled downloader middlewares:
  441. ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
  442. 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
  443. 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
  444. 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
  445. 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
  446. 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
  447. 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
  448. 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
  449. 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
  450. 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
  451. 'scrapy.downloadermiddlewares.stats.DownloaderStats']
  452. 2019-06-22 10:28:43 [scrapy.middleware] INFO: Enabled spider middlewares:
  453. ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
  454. 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
  455. 'scrapy.spidermiddlewares.referer.RefererMiddleware',
  456. 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
  457. 'scrapy.spidermiddlewares.depth.DepthMiddleware']
  458. 2019-06-22 10:28:44 [scrapy.middleware] INFO: Enabled item pipelines:
  459. ['collectSports.pipelines.sportslst.SportslstPipeline']
  460. 2019-06-22 10:28:44 [scrapy.core.engine] INFO: Spider opened
  461. 2019-06-22 10:28:44 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
  462. 2019-06-22 10:28:44 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
  463. 2019-06-22 10:30:25 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
  464. 2019-06-22 10:30:27 [scrapy.core.downloader.tls] WARNING: Ignoring error while verifying certificate from host "205.201.4.177" (exception: ValueError('Invalid DNS-ID.'))
  465. 2019-06-22 10:30:32 [scrapy.utils.log] INFO: Scrapy 1.6.0 started (bot: SportsSpider)
  466. 2019-06-22 10:30:32 [scrapy.utils.log] INFO: Versions: lxml 4.3.2.0, libxml2 2.9.9, cssselect 1.0.3, parsel 1.5.1, w3lib 1.20.0, Twisted 19.2.1, Python 3.7.3 (default, Mar 27 2019, 17:13:21) [MSC v.1915 64 bit (AMD64)], pyOpenSSL 19.0.0 (OpenSSL 1.1.1b 26 Feb 2019), cryptography 2.6.1, Platform Windows-10-10.0.17763-SP0
  467. 2019-06-22 10:30:32 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'SportsSpider', 'LOG_FILE': 'SportsSpiderlog20190622.txt', 'LOG_LEVEL': 'INFO', 'NEWSPIDER_MODULE': 'collectSports.spiders', 'SPIDER_MODULES': ['collectSports.spiders'], 'USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 UBrowser/6.2.4098.3 Safari/537.36'}
  468. 2019-06-22 10:30:32 [scrapy.extensions.telnet] INFO: Telnet Password: 3562166de1a049bb
  469. 2019-06-22 10:30:32 [scrapy.middleware] INFO: Enabled extensions:
  470. ['scrapy.extensions.corestats.CoreStats',
  471. 'scrapy.extensions.telnet.TelnetConsole',
  472. 'scrapy.extensions.logstats.LogStats']
  473. 2019-06-22 10:30:32 [scrapy.middleware] INFO: Enabled downloader middlewares:
  474. ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
  475. 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
  476. 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
  477. 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
  478. 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
  479. 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
  480. 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
  481. 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
  482. 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
  483. 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
  484. 'scrapy.downloadermiddlewares.stats.DownloaderStats']
  485. 2019-06-22 10:30:32 [scrapy.middleware] INFO: Enabled spider middlewares:
  486. ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
  487. 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
  488. 'scrapy.spidermiddlewares.referer.RefererMiddleware',
  489. 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
  490. 'scrapy.spidermiddlewares.depth.DepthMiddleware']
  491. 2019-06-22 10:30:32 [scrapy.middleware] INFO: Enabled item pipelines:
  492. ['collectSports.pipelines.sportslst.SportslstPipeline']
  493. 2019-06-22 10:30:32 [scrapy.core.engine] INFO: Spider opened
  494. 2019-06-22 10:30:32 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
  495. 2019-06-22 10:30:32 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
  496. 2019-06-22 10:30:35 [scrapy.core.downloader.tls] WARNING: Ignoring error while verifying certificate from host "205.201.4.177" (exception: ValueError('Invalid DNS-ID.'))
  497. 2019-06-22 10:31:50 [scrapy.utils.log] INFO: Scrapy 1.6.0 started (bot: SportsSpider)
  498. 2019-06-22 10:31:50 [scrapy.utils.log] INFO: Versions: lxml 4.3.2.0, libxml2 2.9.9, cssselect 1.0.3, parsel 1.5.1, w3lib 1.20.0, Twisted 19.2.1, Python 3.7.3 (default, Mar 27 2019, 17:13:21) [MSC v.1915 64 bit (AMD64)], pyOpenSSL 19.0.0 (OpenSSL 1.1.1b 26 Feb 2019), cryptography 2.6.1, Platform Windows-10-10.0.17763-SP0
  499. 2019-06-22 10:31:50 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'SportsSpider', 'LOG_FILE': 'SportsSpiderlog20190622.txt', 'LOG_LEVEL': 'INFO', 'NEWSPIDER_MODULE': 'collectSports.spiders', 'SPIDER_MODULES': ['collectSports.spiders'], 'USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 UBrowser/6.2.4098.3 Safari/537.36'}
  500. 2019-06-22 10:31:50 [scrapy.extensions.telnet] INFO: Telnet Password: a80b9362da564a2d
  501. 2019-06-22 10:31:50 [scrapy.middleware] INFO: Enabled extensions:
  502. ['scrapy.extensions.corestats.CoreStats',
  503. 'scrapy.extensions.telnet.TelnetConsole',
  504. 'scrapy.extensions.logstats.LogStats']
  505. 2019-06-22 10:31:50 [scrapy.middleware] INFO: Enabled downloader middlewares:
  506. ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
  507. 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
  508. 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
  509. 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
  510. 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
  511. 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
  512. 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
  513. 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
  514. 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
  515. 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
  516. 'scrapy.downloadermiddlewares.stats.DownloaderStats']
  517. 2019-06-22 10:31:50 [scrapy.middleware] INFO: Enabled spider middlewares:
  518. ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
  519. 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
  520. 'scrapy.spidermiddlewares.referer.RefererMiddleware',
  521. 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
  522. 'scrapy.spidermiddlewares.depth.DepthMiddleware']
  523. 2019-06-22 10:31:51 [scrapy.middleware] INFO: Enabled item pipelines:
  524. ['collectSports.pipelines.sportslst.SportslstPipeline']
  525. 2019-06-22 10:31:51 [scrapy.core.engine] INFO: Spider opened
  526. 2019-06-22 10:31:51 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
  527. 2019-06-22 10:31:51 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
  528. 2019-06-22 10:32:22 [scrapy.utils.log] INFO: Scrapy 1.6.0 started (bot: SportsSpider)
  529. 2019-06-22 10:32:22 [scrapy.utils.log] INFO: Versions: lxml 4.3.2.0, libxml2 2.9.9, cssselect 1.0.3, parsel 1.5.1, w3lib 1.20.0, Twisted 19.2.1, Python 3.7.3 (default, Mar 27 2019, 17:13:21) [MSC v.1915 64 bit (AMD64)], pyOpenSSL 19.0.0 (OpenSSL 1.1.1b 26 Feb 2019), cryptography 2.6.1, Platform Windows-10-10.0.17763-SP0
  530. 2019-06-22 10:32:22 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'SportsSpider', 'LOG_FILE': 'SportsSpiderlog20190622.txt', 'LOG_LEVEL': 'INFO', 'NEWSPIDER_MODULE': 'collectSports.spiders', 'SPIDER_MODULES': ['collectSports.spiders'], 'USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 UBrowser/6.2.4098.3 Safari/537.36'}
  531. 2019-06-22 10:32:22 [scrapy.extensions.telnet] INFO: Telnet Password: e0063c1c4b2a77a8
  532. 2019-06-22 10:32:22 [scrapy.middleware] INFO: Enabled extensions:
  533. ['scrapy.extensions.corestats.CoreStats',
  534. 'scrapy.extensions.telnet.TelnetConsole',
  535. 'scrapy.extensions.logstats.LogStats']
  536. 2019-06-22 10:32:22 [scrapy.middleware] INFO: Enabled downloader middlewares:
  537. ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
  538. 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
  539. 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
  540. 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
  541. 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
  542. 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
  543. 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
  544. 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
  545. 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
  546. 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
  547. 'scrapy.downloadermiddlewares.stats.DownloaderStats']
  548. 2019-06-22 10:32:22 [scrapy.middleware] INFO: Enabled spider middlewares:
  549. ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
  550. 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
  551. 'scrapy.spidermiddlewares.referer.RefererMiddleware',
  552. 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
  553. 'scrapy.spidermiddlewares.depth.DepthMiddleware']
  554. 2019-06-22 10:32:22 [scrapy.middleware] INFO: Enabled item pipelines:
  555. ['collectSports.pipelines.sportslst.SportslstPipeline']
  556. 2019-06-22 10:32:22 [scrapy.core.engine] INFO: Spider opened
  557. 2019-06-22 10:32:22 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
  558. 2019-06-22 10:32:22 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
  559. 2019-06-22 10:34:24 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
  560. 2019-06-22 10:34:26 [scrapy.core.downloader.tls] WARNING: Ignoring error while verifying certificate from host "205.201.4.177" (exception: ValueError('Invalid DNS-ID.'))
  561. 2019-06-22 10:35:45 [scrapy.utils.log] INFO: Scrapy 1.6.0 started (bot: SportsSpider)
  562. 2019-06-22 10:35:45 [scrapy.utils.log] INFO: Versions: lxml 4.3.2.0, libxml2 2.9.9, cssselect 1.0.3, parsel 1.5.1, w3lib 1.20.0, Twisted 19.2.1, Python 3.7.3 (default, Mar 27 2019, 17:13:21) [MSC v.1915 64 bit (AMD64)], pyOpenSSL 19.0.0 (OpenSSL 1.1.1b 26 Feb 2019), cryptography 2.6.1, Platform Windows-10-10.0.17763-SP0
  563. 2019-06-22 10:35:45 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'SportsSpider', 'LOG_FILE': 'SportsSpiderlog20190622.txt', 'LOG_LEVEL': 'INFO', 'NEWSPIDER_MODULE': 'collectSports.spiders', 'SPIDER_MODULES': ['collectSports.spiders'], 'USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 UBrowser/6.2.4098.3 Safari/537.36'}
  564. 2019-06-22 10:35:45 [scrapy.extensions.telnet] INFO: Telnet Password: f0012371dcfde2ce
  565. 2019-06-22 10:35:45 [scrapy.middleware] INFO: Enabled extensions:
  566. ['scrapy.extensions.corestats.CoreStats',
  567. 'scrapy.extensions.telnet.TelnetConsole',
  568. 'scrapy.extensions.logstats.LogStats']
  569. 2019-06-22 10:35:45 [scrapy.middleware] INFO: Enabled downloader middlewares:
  570. ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
  571. 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
  572. 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
  573. 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
  574. 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
  575. 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
  576. 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
  577. 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
  578. 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
  579. 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
  580. 'scrapy.downloadermiddlewares.stats.DownloaderStats']
  581. 2019-06-22 10:35:45 [scrapy.middleware] INFO: Enabled spider middlewares:
  582. ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
  583. 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
  584. 'scrapy.spidermiddlewares.referer.RefererMiddleware',
  585. 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
  586. 'scrapy.spidermiddlewares.depth.DepthMiddleware']
  587. 2019-06-22 10:35:45 [scrapy.middleware] INFO: Enabled item pipelines:
  588. ['collectSports.pipelines.sportslst.SportslstPipeline']
  589. 2019-06-22 10:35:45 [scrapy.core.engine] INFO: Spider opened
  590. 2019-06-22 10:35:45 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
  591. 2019-06-22 10:35:45 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
  592. 2019-06-22 10:35:49 [scrapy.core.downloader.tls] WARNING: Ignoring error while verifying certificate from host "199.26.100.178" (exception: ValueError('Invalid DNS-ID.'))
  593. 2019-06-22 10:37:07 [scrapy.utils.log] INFO: Scrapy 1.6.0 started (bot: SportsSpider)
  594. 2019-06-22 10:37:07 [scrapy.utils.log] INFO: Versions: lxml 4.3.2.0, libxml2 2.9.9, cssselect 1.0.3, parsel 1.5.1, w3lib 1.20.0, Twisted 19.2.1, Python 3.7.3 (default, Mar 27 2019, 17:13:21) [MSC v.1915 64 bit (AMD64)], pyOpenSSL 19.0.0 (OpenSSL 1.1.1b 26 Feb 2019), cryptography 2.6.1, Platform Windows-10-10.0.17763-SP0
  595. 2019-06-22 10:37:07 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'SportsSpider', 'LOG_FILE': 'SportsSpiderlog20190622.txt', 'LOG_LEVEL': 'INFO', 'NEWSPIDER_MODULE': 'collectSports.spiders', 'SPIDER_MODULES': ['collectSports.spiders'], 'USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 UBrowser/6.2.4098.3 Safari/537.36'}
  596. 2019-06-22 10:37:07 [scrapy.extensions.telnet] INFO: Telnet Password: 0bb9e65abadbab2e
  597. 2019-06-22 10:37:07 [scrapy.middleware] INFO: Enabled extensions:
  598. ['scrapy.extensions.corestats.CoreStats',
  599. 'scrapy.extensions.telnet.TelnetConsole',
  600. 'scrapy.extensions.logstats.LogStats']
  601. 2019-06-22 10:37:07 [scrapy.middleware] INFO: Enabled downloader middlewares:
  602. ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
  603. 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
  604. 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
  605. 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
  606. 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
  607. 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
  608. 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
  609. 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
  610. 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
  611. 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
  612. 'scrapy.downloadermiddlewares.stats.DownloaderStats']
  613. 2019-06-22 10:37:07 [scrapy.middleware] INFO: Enabled spider middlewares:
  614. ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
  615. 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
  616. 'scrapy.spidermiddlewares.referer.RefererMiddleware',
  617. 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
  618. 'scrapy.spidermiddlewares.depth.DepthMiddleware']
  619. 2019-06-22 10:37:07 [scrapy.middleware] INFO: Enabled item pipelines:
  620. ['collectSports.pipelines.sportslst.SportslstPipeline']
  621. 2019-06-22 10:37:07 [scrapy.core.engine] INFO: Spider opened
  622. 2019-06-22 10:37:07 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
  623. 2019-06-22 10:37:07 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
  624. 2019-06-22 10:48:43 [scrapy.utils.log] INFO: Scrapy 1.6.0 started (bot: SportsSpider)
  625. 2019-06-22 10:48:43 [scrapy.utils.log] INFO: Versions: lxml 4.3.2.0, libxml2 2.9.9, cssselect 1.0.3, parsel 1.5.1, w3lib 1.20.0, Twisted 19.2.1, Python 3.7.3 (default, Mar 27 2019, 17:13:21) [MSC v.1915 64 bit (AMD64)], pyOpenSSL 19.0.0 (OpenSSL 1.1.1b 26 Feb 2019), cryptography 2.6.1, Platform Windows-10-10.0.17763-SP0
  626. 2019-06-22 10:48:43 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'SportsSpider', 'LOG_FILE': 'SportsSpiderlog20190622.txt', 'LOG_LEVEL': 'INFO', 'NEWSPIDER_MODULE': 'collectSports.spiders', 'SPIDER_MODULES': ['collectSports.spiders'], 'USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 UBrowser/6.2.4098.3 Safari/537.36'}
  627. 2019-06-22 10:48:43 [scrapy.extensions.telnet] INFO: Telnet Password: 4317ff83287baaf1
  628. 2019-06-22 10:48:43 [scrapy.middleware] INFO: Enabled extensions:
  629. ['scrapy.extensions.corestats.CoreStats',
  630. 'scrapy.extensions.telnet.TelnetConsole',
  631. 'scrapy.extensions.logstats.LogStats']
  632. 2019-06-22 10:48:43 [scrapy.middleware] INFO: Enabled downloader middlewares:
  633. ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
  634. 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
  635. 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
  636. 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
  637. 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
  638. 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
  639. 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
  640. 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
  641. 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
  642. 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
  643. 'scrapy.downloadermiddlewares.stats.DownloaderStats']
  644. 2019-06-22 10:48:43 [scrapy.middleware] INFO: Enabled spider middlewares:
  645. ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
  646. 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
  647. 'scrapy.spidermiddlewares.referer.RefererMiddleware',
  648. 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
  649. 'scrapy.spidermiddlewares.depth.DepthMiddleware']
  650. 2019-06-22 10:48:43 [scrapy.middleware] INFO: Enabled item pipelines:
  651. ['collectSports.pipelines.sportslst.SportslstPipeline']
  652. 2019-06-22 10:48:43 [scrapy.core.engine] INFO: Spider opened
  653. 2019-06-22 10:48:43 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
  654. 2019-06-22 10:48:43 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
  655. 2019-06-22 10:49:02 [scrapy.utils.log] INFO: Scrapy 1.6.0 started (bot: SportsSpider)
  656. 2019-06-22 10:49:02 [scrapy.utils.log] INFO: Versions: lxml 4.3.2.0, libxml2 2.9.9, cssselect 1.0.3, parsel 1.5.1, w3lib 1.20.0, Twisted 19.2.1, Python 3.7.3 (default, Mar 27 2019, 17:13:21) [MSC v.1915 64 bit (AMD64)], pyOpenSSL 19.0.0 (OpenSSL 1.1.1b 26 Feb 2019), cryptography 2.6.1, Platform Windows-10-10.0.17763-SP0
  657. 2019-06-22 10:49:02 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'SportsSpider', 'LOG_FILE': 'SportsSpiderlog20190622.txt', 'LOG_LEVEL': 'INFO', 'NEWSPIDER_MODULE': 'collectSports.spiders', 'SPIDER_MODULES': ['collectSports.spiders'], 'USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 UBrowser/6.2.4098.3 Safari/537.36'}
  658. 2019-06-22 10:49:02 [scrapy.extensions.telnet] INFO: Telnet Password: 25bdf773b3876abc
  659. 2019-06-22 10:49:02 [scrapy.middleware] INFO: Enabled extensions:
  660. ['scrapy.extensions.corestats.CoreStats',
  661. 'scrapy.extensions.telnet.TelnetConsole',
  662. 'scrapy.extensions.logstats.LogStats']
  663. 2019-06-22 10:49:03 [scrapy.middleware] INFO: Enabled downloader middlewares:
  664. ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
  665. 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
  666. 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
  667. 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
  668. 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
  669. 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
  670. 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
  671. 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
  672. 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
  673. 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
  674. 'scrapy.downloadermiddlewares.stats.DownloaderStats']
  675. 2019-06-22 10:49:03 [scrapy.middleware] INFO: Enabled spider middlewares:
  676. ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
  677. 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
  678. 'scrapy.spidermiddlewares.referer.RefererMiddleware',
  679. 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
  680. 'scrapy.spidermiddlewares.depth.DepthMiddleware']
  681. 2019-06-22 10:49:03 [scrapy.middleware] INFO: Enabled item pipelines:
  682. ['collectSports.pipelines.sportslst.SportslstPipeline']
  683. 2019-06-22 10:49:03 [scrapy.core.engine] INFO: Spider opened
  684. 2019-06-22 10:49:03 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
  685. 2019-06-22 10:49:03 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
  686. 2019-06-22 10:49:09 [scrapy.core.downloader.tls] WARNING: Ignoring error while verifying certificate from host "199.26.100.178" (exception: ValueError('Invalid DNS-ID.'))
  687. 2019-06-22 10:49:45 [scrapy.utils.log] INFO: Scrapy 1.6.0 started (bot: SportsSpider)
  688. 2019-06-22 10:49:45 [scrapy.utils.log] INFO: Versions: lxml 4.3.2.0, libxml2 2.9.9, cssselect 1.0.3, parsel 1.5.1, w3lib 1.20.0, Twisted 19.2.1, Python 3.7.3 (default, Mar 27 2019, 17:13:21) [MSC v.1915 64 bit (AMD64)], pyOpenSSL 19.0.0 (OpenSSL 1.1.1b 26 Feb 2019), cryptography 2.6.1, Platform Windows-10-10.0.17763-SP0
  689. 2019-06-22 10:49:45 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'SportsSpider', 'LOG_FILE': 'SportsSpiderlog20190622.txt', 'LOG_LEVEL': 'INFO', 'NEWSPIDER_MODULE': 'collectSports.spiders', 'SPIDER_MODULES': ['collectSports.spiders'], 'USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 UBrowser/6.2.4098.3 Safari/537.36'}
  690. 2019-06-22 10:49:45 [scrapy.extensions.telnet] INFO: Telnet Password: 6c328f0307217b75
  691. 2019-06-22 10:49:45 [scrapy.middleware] INFO: Enabled extensions:
  692. ['scrapy.extensions.corestats.CoreStats',
  693. 'scrapy.extensions.telnet.TelnetConsole',
  694. 'scrapy.extensions.logstats.LogStats']
  695. 2019-06-22 10:49:45 [scrapy.middleware] INFO: Enabled downloader middlewares:
  696. ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
  697. 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
  698. 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
  699. 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
  700. 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
  701. 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
  702. 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
  703. 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
  704. 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
  705. 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
  706. 'scrapy.downloadermiddlewares.stats.DownloaderStats']
  707. 2019-06-22 10:49:45 [scrapy.middleware] INFO: Enabled spider middlewares:
  708. ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
  709. 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
  710. 'scrapy.spidermiddlewares.referer.RefererMiddleware',
  711. 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
  712. 'scrapy.spidermiddlewares.depth.DepthMiddleware']
  713. 2019-06-22 10:49:45 [scrapy.middleware] INFO: Enabled item pipelines:
  714. ['collectSports.pipelines.sportslst.SportslstPipeline']
  715. 2019-06-22 10:49:45 [scrapy.core.engine] INFO: Spider opened
  716. 2019-06-22 10:49:45 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
  717. 2019-06-22 10:49:45 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
  718. 2019-06-22 10:49:57 [scrapy.utils.log] INFO: Scrapy 1.6.0 started (bot: SportsSpider)
  719. 2019-06-22 10:49:57 [scrapy.utils.log] INFO: Versions: lxml 4.3.2.0, libxml2 2.9.9, cssselect 1.0.3, parsel 1.5.1, w3lib 1.20.0, Twisted 19.2.1, Python 3.7.3 (default, Mar 27 2019, 17:13:21) [MSC v.1915 64 bit (AMD64)], pyOpenSSL 19.0.0 (OpenSSL 1.1.1b 26 Feb 2019), cryptography 2.6.1, Platform Windows-10-10.0.17763-SP0
  720. 2019-06-22 10:49:57 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'SportsSpider', 'LOG_FILE': 'SportsSpiderlog20190622.txt', 'LOG_LEVEL': 'INFO', 'NEWSPIDER_MODULE': 'collectSports.spiders', 'SPIDER_MODULES': ['collectSports.spiders'], 'USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 UBrowser/6.2.4098.3 Safari/537.36'}
  721. 2019-06-22 10:49:57 [scrapy.extensions.telnet] INFO: Telnet Password: 4fb03eca233bcca3
  722. 2019-06-22 10:49:57 [scrapy.middleware] INFO: Enabled extensions:
  723. ['scrapy.extensions.corestats.CoreStats',
  724. 'scrapy.extensions.telnet.TelnetConsole',
  725. 'scrapy.extensions.logstats.LogStats']
  726. 2019-06-22 10:49:58 [scrapy.middleware] INFO: Enabled downloader middlewares:
  727. ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
  728. 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
  729. 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
  730. 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
  731. 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
  732. 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
  733. 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
  734. 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
  735. 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
  736. 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
  737. 'scrapy.downloadermiddlewares.stats.DownloaderStats']
  738. 2019-06-22 10:49:58 [scrapy.middleware] INFO: Enabled spider middlewares:
  739. ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
  740. 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
  741. 'scrapy.spidermiddlewares.referer.RefererMiddleware',
  742. 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
  743. 'scrapy.spidermiddlewares.depth.DepthMiddleware']
  744. 2019-06-22 10:49:58 [scrapy.middleware] INFO: Enabled item pipelines:
  745. ['collectSports.pipelines.sportslst.SportslstPipeline']
  746. 2019-06-22 10:49:58 [scrapy.core.engine] INFO: Spider opened
  747. 2019-06-22 10:49:58 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
  748. 2019-06-22 10:49:58 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
  749. 2019-06-22 10:50:03 [scrapy.core.downloader.tls] WARNING: Ignoring error while verifying certificate from host "205.201.4.177" (exception: ValueError('Invalid DNS-ID.'))
  750. 2019-06-22 10:50:56 [scrapy.utils.log] INFO: Scrapy 1.6.0 started (bot: SportsSpider)
  751. 2019-06-22 10:50:56 [scrapy.utils.log] INFO: Versions: lxml 4.3.2.0, libxml2 2.9.9, cssselect 1.0.3, parsel 1.5.1, w3lib 1.20.0, Twisted 19.2.1, Python 3.7.3 (default, Mar 27 2019, 17:13:21) [MSC v.1915 64 bit (AMD64)], pyOpenSSL 19.0.0 (OpenSSL 1.1.1b 26 Feb 2019), cryptography 2.6.1, Platform Windows-10-10.0.17763-SP0
  752. 2019-06-22 10:50:56 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'SportsSpider', 'LOG_FILE': 'SportsSpiderlog20190622.txt', 'LOG_LEVEL': 'INFO', 'NEWSPIDER_MODULE': 'collectSports.spiders', 'SPIDER_MODULES': ['collectSports.spiders'], 'USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 UBrowser/6.2.4098.3 Safari/537.36'}
  753. 2019-06-22 10:50:56 [scrapy.extensions.telnet] INFO: Telnet Password: 66cc7c4c24cc3cae
  754. 2019-06-22 10:50:56 [scrapy.middleware] INFO: Enabled extensions:
  755. ['scrapy.extensions.corestats.CoreStats',
  756. 'scrapy.extensions.telnet.TelnetConsole',
  757. 'scrapy.extensions.logstats.LogStats']
  758. 2019-06-22 10:50:56 [scrapy.middleware] INFO: Enabled downloader middlewares:
  759. ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
  760. 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
  761. 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
  762. 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
  763. 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
  764. 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
  765. 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
  766. 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
  767. 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
  768. 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
  769. 'scrapy.downloadermiddlewares.stats.DownloaderStats']
  770. 2019-06-22 10:50:56 [scrapy.middleware] INFO: Enabled spider middlewares:
  771. ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
  772. 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
  773. 'scrapy.spidermiddlewares.referer.RefererMiddleware',
  774. 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
  775. 'scrapy.spidermiddlewares.depth.DepthMiddleware']
  776. 2019-06-22 10:50:56 [scrapy.middleware] INFO: Enabled item pipelines:
  777. ['collectSports.pipelines.sportslst.SportslstPipeline']
  778. 2019-06-22 10:50:56 [scrapy.core.engine] INFO: Spider opened
  779. 2019-06-22 10:50:56 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
  780. 2019-06-22 10:50:56 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
  781. 2019-06-22 10:50:59 [scrapy.core.downloader.tls] WARNING: Ignoring error while verifying certificate from host "205.201.4.177" (exception: ValueError('Invalid DNS-ID.'))
  782. 2019-06-22 11:04:38 [scrapy.utils.log] INFO: Scrapy 1.6.0 started (bot: SportsSpider)
  783. 2019-06-22 11:04:38 [scrapy.utils.log] INFO: Versions: lxml 4.3.2.0, libxml2 2.9.9, cssselect 1.0.3, parsel 1.5.1, w3lib 1.20.0, Twisted 19.2.1, Python 3.7.3 (default, Mar 27 2019, 17:13:21) [MSC v.1915 64 bit (AMD64)], pyOpenSSL 19.0.0 (OpenSSL 1.1.1b 26 Feb 2019), cryptography 2.6.1, Platform Windows-10-10.0.17763-SP0
  784. 2019-06-22 11:04:38 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'SportsSpider', 'LOG_FILE': 'SportsSpiderlog20190622.txt', 'LOG_LEVEL': 'INFO', 'NEWSPIDER_MODULE': 'collectSports.spiders', 'SPIDER_MODULES': ['collectSports.spiders'], 'USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 UBrowser/6.2.4098.3 Safari/537.36'}
  785. 2019-06-22 11:04:38 [scrapy.extensions.telnet] INFO: Telnet Password: b810244ec17ecefd
  786. 2019-06-22 11:04:39 [scrapy.middleware] INFO: Enabled extensions:
  787. ['scrapy.extensions.corestats.CoreStats',
  788. 'scrapy.extensions.telnet.TelnetConsole',
  789. 'scrapy.extensions.logstats.LogStats']
  790. 2019-06-22 11:04:39 [scrapy.middleware] INFO: Enabled downloader middlewares:
  791. ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
  792. 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
  793. 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
  794. 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
  795. 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
  796. 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
  797. 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
  798. 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
  799. 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
  800. 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
  801. 'scrapy.downloadermiddlewares.stats.DownloaderStats']
  802. 2019-06-22 11:04:39 [scrapy.middleware] INFO: Enabled spider middlewares:
  803. ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
  804. 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
  805. 'scrapy.spidermiddlewares.referer.RefererMiddleware',
  806. 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
  807. 'scrapy.spidermiddlewares.depth.DepthMiddleware']
  808. 2019-06-22 11:04:39 [scrapy.middleware] INFO: Enabled item pipelines:
  809. ['collectSports.pipelines.sportslst.SportslstPipeline']
  810. 2019-06-22 11:04:39 [scrapy.core.engine] INFO: Spider opened
  811. 2019-06-22 11:04:39 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
  812. 2019-06-22 11:04:39 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
  813. 2019-06-22 11:05:41 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
  814. 2019-06-22 11:05:41 [scrapy.core.downloader.tls] WARNING: Ignoring error while verifying certificate from host "205.201.4.177" (exception: ValueError('Invalid DNS-ID.'))
  815. 2019-06-22 11:09:12 [scrapy.utils.log] INFO: Scrapy 1.6.0 started (bot: SportsSpider)
  816. 2019-06-22 11:09:12 [scrapy.utils.log] INFO: Versions: lxml 4.3.2.0, libxml2 2.9.9, cssselect 1.0.3, parsel 1.5.1, w3lib 1.20.0, Twisted 19.2.1, Python 3.7.3 (default, Mar 27 2019, 17:13:21) [MSC v.1915 64 bit (AMD64)], pyOpenSSL 19.0.0 (OpenSSL 1.1.1b 26 Feb 2019), cryptography 2.6.1, Platform Windows-10-10.0.17763-SP0
  817. 2019-06-22 11:09:12 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'SportsSpider', 'LOG_FILE': 'SportsSpiderlog20190622.txt', 'LOG_LEVEL': 'INFO', 'NEWSPIDER_MODULE': 'collectSports.spiders', 'SPIDER_MODULES': ['collectSports.spiders'], 'USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 UBrowser/6.2.4098.3 Safari/537.36'}
  818. 2019-06-22 11:09:12 [scrapy.extensions.telnet] INFO: Telnet Password: 7def27b7bb4424ee
  819. 2019-06-22 11:09:12 [scrapy.middleware] INFO: Enabled extensions:
  820. ['scrapy.extensions.corestats.CoreStats',
  821. 'scrapy.extensions.telnet.TelnetConsole',
  822. 'scrapy.extensions.logstats.LogStats']
  823. 2019-06-22 11:09:12 [scrapy.middleware] INFO: Enabled downloader middlewares:
  824. ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
  825. 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
  826. 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
  827. 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
  828. 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
  829. 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
  830. 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
  831. 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
  832. 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
  833. 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
  834. 'scrapy.downloadermiddlewares.stats.DownloaderStats']
  835. 2019-06-22 11:09:12 [scrapy.middleware] INFO: Enabled spider middlewares:
  836. ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
  837. 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
  838. 'scrapy.spidermiddlewares.referer.RefererMiddleware',
  839. 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
  840. 'scrapy.spidermiddlewares.depth.DepthMiddleware']
  841. 2019-06-22 11:09:12 [scrapy.middleware] INFO: Enabled item pipelines:
  842. ['collectSports.pipelines.sportslst.SportslstPipeline']
  843. 2019-06-22 11:09:12 [scrapy.core.engine] INFO: Spider opened
  844. 2019-06-22 11:09:12 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
  845. 2019-06-22 11:09:12 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
  846. 2019-06-22 11:09:16 [scrapy.core.downloader.tls] WARNING: Ignoring error while verifying certificate from host "205.201.4.177" (exception: ValueError('Invalid DNS-ID.'))
  847. 2019-06-22 11:12:01 [scrapy.utils.log] INFO: Scrapy 1.6.0 started (bot: SportsSpider)
  848. 2019-06-22 11:12:01 [scrapy.utils.log] INFO: Versions: lxml 4.3.2.0, libxml2 2.9.9, cssselect 1.0.3, parsel 1.5.1, w3lib 1.20.0, Twisted 19.2.1, Python 3.7.3 (default, Mar 27 2019, 17:13:21) [MSC v.1915 64 bit (AMD64)], pyOpenSSL 19.0.0 (OpenSSL 1.1.1b 26 Feb 2019), cryptography 2.6.1, Platform Windows-10-10.0.17763-SP0
  849. 2019-06-22 11:12:01 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'SportsSpider', 'LOG_FILE': 'SportsSpiderlog20190622.txt', 'LOG_LEVEL': 'INFO', 'NEWSPIDER_MODULE': 'collectSports.spiders', 'SPIDER_MODULES': ['collectSports.spiders'], 'USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 UBrowser/6.2.4098.3 Safari/537.36'}
  850. 2019-06-22 11:12:01 [scrapy.extensions.telnet] INFO: Telnet Password: 1985e25100bae467
  851. 2019-06-22 11:12:01 [scrapy.middleware] INFO: Enabled extensions:
  852. ['scrapy.extensions.corestats.CoreStats',
  853. 'scrapy.extensions.telnet.TelnetConsole',
  854. 'scrapy.extensions.logstats.LogStats']
  855. 2019-06-22 11:12:01 [scrapy.middleware] INFO: Enabled downloader middlewares:
  856. ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
  857. 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
  858. 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
  859. 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
  860. 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
  861. 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
  862. 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
  863. 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
  864. 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
  865. 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
  866. 'scrapy.downloadermiddlewares.stats.DownloaderStats']
  867. 2019-06-22 11:12:01 [scrapy.middleware] INFO: Enabled spider middlewares:
  868. ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
  869. 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
  870. 'scrapy.spidermiddlewares.referer.RefererMiddleware',
  871. 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
  872. 'scrapy.spidermiddlewares.depth.DepthMiddleware']
  873. 2019-06-22 11:12:01 [scrapy.middleware] INFO: Enabled item pipelines:
  874. ['collectSports.pipelines.sportslst.SportslstPipeline']
  875. 2019-06-22 11:12:01 [scrapy.core.engine] INFO: Spider opened
  876. 2019-06-22 11:12:01 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
  877. 2019-06-22 11:12:01 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
  878. 2019-06-22 11:12:11 [scrapy.core.downloader.tls] WARNING: Ignoring error while verifying certificate from host "199.26.100.178" (exception: ValueError('Invalid DNS-ID.'))
  879. 2019-06-22 11:31:13 [scrapy.utils.log] INFO: Scrapy 1.6.0 started (bot: SportsSpider)
  880. 2019-06-22 11:31:13 [scrapy.utils.log] INFO: Versions: lxml 4.3.2.0, libxml2 2.9.9, cssselect 1.0.3, parsel 1.5.1, w3lib 1.20.0, Twisted 19.2.1, Python 3.7.3 (default, Mar 27 2019, 17:13:21) [MSC v.1915 64 bit (AMD64)], pyOpenSSL 19.0.0 (OpenSSL 1.1.1b 26 Feb 2019), cryptography 2.6.1, Platform Windows-10-10.0.17763-SP0
  881. 2019-06-22 11:31:13 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'SportsSpider', 'LOG_FILE': 'SportsSpiderlog20190622.txt', 'LOG_LEVEL': 'INFO', 'NEWSPIDER_MODULE': 'collectSports.spiders', 'SPIDER_MODULES': ['collectSports.spiders'], 'USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 UBrowser/6.2.4098.3 Safari/537.36'}
  882. 2019-06-22 11:31:13 [scrapy.extensions.telnet] INFO: Telnet Password: 62900213ae5d40a0
  883. 2019-06-22 11:31:13 [scrapy.middleware] INFO: Enabled extensions:
  884. ['scrapy.extensions.corestats.CoreStats',
  885. 'scrapy.extensions.telnet.TelnetConsole',
  886. 'scrapy.extensions.logstats.LogStats']
  887. 2019-06-22 11:31:14 [scrapy.middleware] INFO: Enabled downloader middlewares:
  888. ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
  889. 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
  890. 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
  891. 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
  892. 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
  893. 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
  894. 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
  895. 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
  896. 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
  897. 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
  898. 'scrapy.downloadermiddlewares.stats.DownloaderStats']
  899. 2019-06-22 11:31:14 [scrapy.middleware] INFO: Enabled spider middlewares:
  900. ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
  901. 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
  902. 'scrapy.spidermiddlewares.referer.RefererMiddleware',
  903. 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
  904. 'scrapy.spidermiddlewares.depth.DepthMiddleware']
  905. 2019-06-22 11:31:14 [scrapy.middleware] INFO: Enabled item pipelines:
  906. ['collectSports.pipelines.sportslst.SportslstPipeline']
  907. 2019-06-22 11:31:14 [scrapy.core.engine] INFO: Spider opened
  908. 2019-06-22 11:31:14 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
  909. 2019-06-22 11:31:14 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
  910. 2019-06-22 11:31:18 [scrapy.core.downloader.tls] WARNING: Ignoring error while verifying certificate from host "205.201.4.177" (exception: ValueError('Invalid DNS-ID.'))
  911. 2019-06-22 11:31:46 [scrapy.core.engine] INFO: Closing spider (finished)
  912. 2019-06-22 11:31:46 [scrapy.core.engine] ERROR: Scraper close failure
  913. Traceback (most recent call last):
  914. File "C:\ProgramData\Anaconda3\lib\site-packages\twisted\internet\defer.py", line 654, in _runCallbacks
  915. current.result = callback(current.result, *args, **kw)
  916. File "D:\workspace\sports_collect\collectSports\pipelines\sportslst.py", line 170, in close_spider
  917. self.client.close()
  918. AttributeError: 'SportslstPipeline' object has no attribute 'client'
  919. 2019-06-22 11:31:46 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
  920. {'downloader/request_bytes': 455,
  921. 'downloader/request_count': 1,
  922. 'downloader/request_method_count/GET': 1,
  923. 'downloader/response_bytes': 4730,
  924. 'downloader/response_count': 1,
  925. 'downloader/response_status_count/200': 1,
  926. 'finish_reason': 'finished',
  927. 'finish_time': datetime.datetime(2019, 6, 22, 3, 31, 46, 465865),
  928. 'item_scraped_count': 1,
  929. 'log_count/ERROR': 1,
  930. 'log_count/INFO': 9,
  931. 'log_count/WARNING': 1,
  932. 'response_received_count': 1,
  933. 'scheduler/dequeued': 1,
  934. 'scheduler/dequeued/memory': 1,
  935. 'scheduler/enqueued': 1,
  936. 'scheduler/enqueued/memory': 1,
  937. 'start_time': datetime.datetime(2019, 6, 22, 3, 31, 14, 150055)}
  938. 2019-06-22 11:31:46 [scrapy.core.engine] INFO: Spider closed (finished)
  939. 2019-06-22 11:32:05 [scrapy.utils.log] INFO: Scrapy 1.6.0 started (bot: SportsSpider)
  940. 2019-06-22 11:32:05 [scrapy.utils.log] INFO: Versions: lxml 4.3.2.0, libxml2 2.9.9, cssselect 1.0.3, parsel 1.5.1, w3lib 1.20.0, Twisted 19.2.1, Python 3.7.3 (default, Mar 27 2019, 17:13:21) [MSC v.1915 64 bit (AMD64)], pyOpenSSL 19.0.0 (OpenSSL 1.1.1b 26 Feb 2019), cryptography 2.6.1, Platform Windows-10-10.0.17763-SP0
  941. 2019-06-22 11:32:05 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'SportsSpider', 'LOG_FILE': 'SportsSpiderlog20190622.txt', 'LOG_LEVEL': 'INFO', 'NEWSPIDER_MODULE': 'collectSports.spiders', 'SPIDER_MODULES': ['collectSports.spiders'], 'USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 UBrowser/6.2.4098.3 Safari/537.36'}
  942. 2019-06-22 11:32:05 [scrapy.extensions.telnet] INFO: Telnet Password: 70a29d0d9071d1a7
  943. 2019-06-22 11:32:05 [scrapy.middleware] INFO: Enabled extensions:
  944. ['scrapy.extensions.corestats.CoreStats',
  945. 'scrapy.extensions.telnet.TelnetConsole',
  946. 'scrapy.extensions.logstats.LogStats']
  947. 2019-06-22 11:32:06 [scrapy.middleware] INFO: Enabled downloader middlewares:
  948. ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
  949. 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
  950. 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
  951. 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
  952. 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
  953. 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
  954. 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
  955. 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
  956. 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
  957. 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
  958. 'scrapy.downloadermiddlewares.stats.DownloaderStats']
  959. 2019-06-22 11:32:06 [scrapy.middleware] INFO: Enabled spider middlewares:
  960. ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
  961. 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
  962. 'scrapy.spidermiddlewares.referer.RefererMiddleware',
  963. 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
  964. 'scrapy.spidermiddlewares.depth.DepthMiddleware']
  965. 2019-06-22 11:32:06 [scrapy.middleware] INFO: Enabled item pipelines:
  966. ['collectSports.pipelines.sportslst.SportslstPipeline']
  967. 2019-06-22 11:32:06 [scrapy.core.engine] INFO: Spider opened
  968. 2019-06-22 11:32:06 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
  969. 2019-06-22 11:32:06 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
  970. 2019-06-22 11:32:06 [scrapy.core.downloader.tls] WARNING: Ignoring error while verifying certificate from host "205.201.4.177" (exception: ValueError('Invalid DNS-ID.'))
  971. 2019-06-22 11:32:11 [scrapy.core.engine] INFO: Closing spider (finished)
  972. 2019-06-22 11:32:11 [scrapy.core.engine] ERROR: Scraper close failure
  973. Traceback (most recent call last):
  974. File "C:\ProgramData\Anaconda3\lib\site-packages\twisted\internet\defer.py", line 654, in _runCallbacks
  975. current.result = callback(current.result, *args, **kw)
  976. File "D:\workspace\sports_collect\collectSports\pipelines\sportslst.py", line 170, in close_spider
  977. self.client.close()
  978. AttributeError: 'SportslstPipeline' object has no attribute 'client'
  979. 2019-06-22 11:32:11 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
  980. {'downloader/request_bytes': 455,
  981. 'downloader/request_count': 1,
  982. 'downloader/request_method_count/GET': 1,
  983. 'downloader/response_bytes': 4731,
  984. 'downloader/response_count': 1,
  985. 'downloader/response_status_count/200': 1,
  986. 'finish_reason': 'finished',
  987. 'finish_time': datetime.datetime(2019, 6, 22, 3, 32, 11, 335988),
  988. 'item_scraped_count': 1,
  989. 'log_count/ERROR': 1,
  990. 'log_count/INFO': 9,
  991. 'log_count/WARNING': 1,
  992. 'response_received_count': 1,
  993. 'scheduler/dequeued': 1,
  994. 'scheduler/dequeued/memory': 1,
  995. 'scheduler/enqueued': 1,
  996. 'scheduler/enqueued/memory': 1,
  997. 'start_time': datetime.datetime(2019, 6, 22, 3, 32, 6, 130610)}
  998. 2019-06-22 11:32:11 [scrapy.core.engine] INFO: Spider closed (finished)
  999. 2019-06-22 11:32:27 [scrapy.utils.log] INFO: Scrapy 1.6.0 started (bot: SportsSpider)
  1000. 2019-06-22 11:32:27 [scrapy.utils.log] INFO: Versions: lxml 4.3.2.0, libxml2 2.9.9, cssselect 1.0.3, parsel 1.5.1, w3lib 1.20.0, Twisted 19.2.1, Python 3.7.3 (default, Mar 27 2019, 17:13:21) [MSC v.1915 64 bit (AMD64)], pyOpenSSL 19.0.0 (OpenSSL 1.1.1b 26 Feb 2019), cryptography 2.6.1, Platform Windows-10-10.0.17763-SP0
  1001. 2019-06-22 11:32:27 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'SportsSpider', 'LOG_FILE': 'SportsSpiderlog20190622.txt', 'LOG_LEVEL': 'INFO', 'NEWSPIDER_MODULE': 'collectSports.spiders', 'SPIDER_MODULES': ['collectSports.spiders'], 'USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 UBrowser/6.2.4098.3 Safari/537.36'}
  1002. 2019-06-22 11:32:27 [scrapy.extensions.telnet] INFO: Telnet Password: 64b2924bd6b7e99d
  1003. 2019-06-22 11:32:27 [scrapy.middleware] INFO: Enabled extensions:
  1004. ['scrapy.extensions.corestats.CoreStats',
  1005. 'scrapy.extensions.telnet.TelnetConsole',
  1006. 'scrapy.extensions.logstats.LogStats']
  1007. 2019-06-22 11:32:27 [scrapy.middleware] INFO: Enabled downloader middlewares:
  1008. ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
  1009. 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
  1010. 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
  1011. 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
  1012. 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
  1013. 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
  1014. 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
  1015. 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
  1016. 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
  1017. 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
  1018. 'scrapy.downloadermiddlewares.stats.DownloaderStats']
  1019. 2019-06-22 11:32:27 [scrapy.middleware] INFO: Enabled spider middlewares:
  1020. ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
  1021. 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
  1022. 'scrapy.spidermiddlewares.referer.RefererMiddleware',
  1023. 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
  1024. 'scrapy.spidermiddlewares.depth.DepthMiddleware']
  1025. 2019-06-22 11:32:27 [scrapy.middleware] INFO: Enabled item pipelines:
  1026. ['collectSports.pipelines.sportslst.SportslstPipeline']
  1027. 2019-06-22 11:32:27 [scrapy.core.engine] INFO: Spider opened
  1028. 2019-06-22 11:32:27 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
  1029. 2019-06-22 11:32:27 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
  1030. 2019-06-22 11:32:28 [scrapy.core.downloader.tls] WARNING: Ignoring error while verifying certificate from host "205.201.4.177" (exception: ValueError('Invalid DNS-ID.'))
  1031. 2019-06-22 11:34:48 [scrapy.extensions.logstats] INFO: Crawled 1 pages (at 1 pages/min), scraped 1 items (at 1 items/min)
  1032. 2019-06-22 11:34:48 [scrapy.core.engine] INFO: Closing spider (finished)
  1033. 2019-06-22 11:34:48 [scrapy.core.engine] ERROR: Scraper close failure
  1034. Traceback (most recent call last):
  1035. File "C:\ProgramData\Anaconda3\lib\site-packages\twisted\internet\defer.py", line 654, in _runCallbacks
  1036. current.result = callback(current.result, *args, **kw)
  1037. File "D:\workspace\sports_collect\collectSports\pipelines\sportslst.py", line 170, in close_spider
  1038. self.client.close()
  1039. AttributeError: 'SportslstPipeline' object has no attribute 'client'
  1040. 2019-06-22 11:34:48 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
  1041. {'downloader/request_bytes': 455,
  1042. 'downloader/request_count': 1,
  1043. 'downloader/request_method_count/GET': 1,
  1044. 'downloader/response_bytes': 4729,
  1045. 'downloader/response_count': 1,
  1046. 'downloader/response_status_count/200': 1,
  1047. 'finish_reason': 'finished',
  1048. 'finish_time': datetime.datetime(2019, 6, 22, 3, 34, 48, 972961),
  1049. 'item_scraped_count': 1,
  1050. 'log_count/ERROR': 1,
  1051. 'log_count/INFO': 10,
  1052. 'log_count/WARNING': 1,
  1053. 'response_received_count': 1,
  1054. 'scheduler/dequeued': 1,
  1055. 'scheduler/dequeued/memory': 1,
  1056. 'scheduler/enqueued': 1,
  1057. 'scheduler/enqueued/memory': 1,
  1058. 'start_time': datetime.datetime(2019, 6, 22, 3, 32, 27, 913085)}
  1059. 2019-06-22 11:34:48 [scrapy.core.engine] INFO: Spider closed (finished)
  1060. 2019-06-22 11:35:10 [scrapy.utils.log] INFO: Scrapy 1.6.0 started (bot: SportsSpider)
  1061. 2019-06-22 11:35:10 [scrapy.utils.log] INFO: Versions: lxml 4.3.2.0, libxml2 2.9.9, cssselect 1.0.3, parsel 1.5.1, w3lib 1.20.0, Twisted 19.2.1, Python 3.7.3 (default, Mar 27 2019, 17:13:21) [MSC v.1915 64 bit (AMD64)], pyOpenSSL 19.0.0 (OpenSSL 1.1.1b 26 Feb 2019), cryptography 2.6.1, Platform Windows-10-10.0.17763-SP0
  1062. 2019-06-22 11:35:10 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'SportsSpider', 'LOG_FILE': 'SportsSpiderlog20190622.txt', 'LOG_LEVEL': 'INFO', 'NEWSPIDER_MODULE': 'collectSports.spiders', 'SPIDER_MODULES': ['collectSports.spiders'], 'USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 UBrowser/6.2.4098.3 Safari/537.36'}
  1063. 2019-06-22 11:35:10 [scrapy.extensions.telnet] INFO: Telnet Password: c93ed231f5d9ff4c
  1064. 2019-06-22 11:35:10 [scrapy.middleware] INFO: Enabled extensions:
  1065. ['scrapy.extensions.corestats.CoreStats',
  1066. 'scrapy.extensions.telnet.TelnetConsole',
  1067. 'scrapy.extensions.logstats.LogStats']
  1068. 2019-06-22 11:35:10 [scrapy.middleware] INFO: Enabled downloader middlewares:
  1069. ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
  1070. 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
  1071. 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
  1072. 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
  1073. 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
  1074. 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
  1075. 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
  1076. 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
  1077. 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
  1078. 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
  1079. 'scrapy.downloadermiddlewares.stats.DownloaderStats']
  1080. 2019-06-22 11:35:10 [scrapy.middleware] INFO: Enabled spider middlewares:
  1081. ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
  1082. 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
  1083. 'scrapy.spidermiddlewares.referer.RefererMiddleware',
  1084. 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
  1085. 'scrapy.spidermiddlewares.depth.DepthMiddleware']
  1086. 2019-06-22 11:35:10 [scrapy.middleware] INFO: Enabled item pipelines:
  1087. ['collectSports.pipelines.sportslst.SportslstPipeline']
  1088. 2019-06-22 11:35:10 [scrapy.core.engine] INFO: Spider opened
  1089. 2019-06-22 11:35:10 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
  1090. 2019-06-22 11:35:10 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
  1091. 2019-06-22 11:35:10 [scrapy.core.downloader.tls] WARNING: Ignoring error while verifying certificate from host "205.201.4.177" (exception: ValueError('Invalid DNS-ID.'))
  1092. 2019-06-22 11:36:05 [scrapy.utils.log] INFO: Scrapy 1.6.0 started (bot: SportsSpider)
  1093. 2019-06-22 11:36:05 [scrapy.utils.log] INFO: Versions: lxml 4.3.2.0, libxml2 2.9.9, cssselect 1.0.3, parsel 1.5.1, w3lib 1.20.0, Twisted 19.2.1, Python 3.7.3 (default, Mar 27 2019, 17:13:21) [MSC v.1915 64 bit (AMD64)], pyOpenSSL 19.0.0 (OpenSSL 1.1.1b 26 Feb 2019), cryptography 2.6.1, Platform Windows-10-10.0.17763-SP0
  1094. 2019-06-22 11:36:05 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'SportsSpider', 'LOG_FILE': 'SportsSpiderlog20190622.txt', 'LOG_LEVEL': 'INFO', 'NEWSPIDER_MODULE': 'collectSports.spiders', 'SPIDER_MODULES': ['collectSports.spiders'], 'USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 UBrowser/6.2.4098.3 Safari/537.36'}
  1095. 2019-06-22 11:36:05 [scrapy.extensions.telnet] INFO: Telnet Password: c52ded6dec493710
  1096. 2019-06-22 11:36:05 [scrapy.middleware] INFO: Enabled extensions:
  1097. ['scrapy.extensions.corestats.CoreStats',
  1098. 'scrapy.extensions.telnet.TelnetConsole',
  1099. 'scrapy.extensions.logstats.LogStats']
  1100. 2019-06-22 11:36:05 [scrapy.middleware] INFO: Enabled downloader middlewares:
  1101. ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
  1102. 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
  1103. 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
  1104. 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
  1105. 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
  1106. 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
  1107. 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
  1108. 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
  1109. 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
  1110. 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
  1111. 'scrapy.downloadermiddlewares.stats.DownloaderStats']
  1112. 2019-06-22 11:36:05 [scrapy.middleware] INFO: Enabled spider middlewares:
  1113. ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
  1114. 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
  1115. 'scrapy.spidermiddlewares.referer.RefererMiddleware',
  1116. 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
  1117. 'scrapy.spidermiddlewares.depth.DepthMiddleware']
  1118. 2019-06-22 11:36:05 [scrapy.middleware] INFO: Enabled item pipelines:
  1119. ['collectSports.pipelines.sportslst.SportslstPipeline']
  1120. 2019-06-22 11:36:05 [scrapy.core.engine] INFO: Spider opened
  1121. 2019-06-22 11:36:05 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
  1122. 2019-06-22 11:36:05 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
  1123. 2019-06-22 11:36:05 [scrapy.core.downloader.tls] WARNING: Ignoring error while verifying certificate from host "205.201.4.177" (exception: ValueError('Invalid DNS-ID.'))
  1124. 2019-06-22 11:36:10 [scrapy.core.engine] INFO: Closing spider (finished)
  1125. 2019-06-22 11:36:10 [scrapy.core.engine] ERROR: Scraper close failure
  1126. Traceback (most recent call last):
  1127. File "C:\ProgramData\Anaconda3\lib\site-packages\twisted\internet\defer.py", line 654, in _runCallbacks
  1128. current.result = callback(current.result, *args, **kw)
  1129. File "D:\workspace\sports_collect\collectSports\pipelines\sportslst.py", line 170, in close_spider
  1130. self.client.close()
  1131. AttributeError: 'SportslstPipeline' object has no attribute 'client'
  1132. 2019-06-22 11:36:10 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
  1133. {'downloader/request_bytes': 455,
  1134. 'downloader/request_count': 1,
  1135. 'downloader/request_method_count/GET': 1,
  1136. 'downloader/response_bytes': 4728,
  1137. 'downloader/response_count': 1,
  1138. 'downloader/response_status_count/200': 1,
  1139. 'finish_reason': 'finished',
  1140. 'finish_time': datetime.datetime(2019, 6, 22, 3, 36, 10, 724928),
  1141. 'item_scraped_count': 1,
  1142. 'log_count/ERROR': 1,
  1143. 'log_count/INFO': 9,
  1144. 'log_count/WARNING': 1,
  1145. 'response_received_count': 1,
  1146. 'scheduler/dequeued': 1,
  1147. 'scheduler/dequeued/memory': 1,
  1148. 'scheduler/enqueued': 1,
  1149. 'scheduler/enqueued/memory': 1,
  1150. 'start_time': datetime.datetime(2019, 6, 22, 3, 36, 5, 426056)}
  1151. 2019-06-22 11:36:10 [scrapy.core.engine] INFO: Spider closed (finished)
  1152. 2019-06-22 11:36:27 [scrapy.utils.log] INFO: Scrapy 1.6.0 started (bot: SportsSpider)
  1153. 2019-06-22 11:36:27 [scrapy.utils.log] INFO: Versions: lxml 4.3.2.0, libxml2 2.9.9, cssselect 1.0.3, parsel 1.5.1, w3lib 1.20.0, Twisted 19.2.1, Python 3.7.3 (default, Mar 27 2019, 17:13:21) [MSC v.1915 64 bit (AMD64)], pyOpenSSL 19.0.0 (OpenSSL 1.1.1b 26 Feb 2019), cryptography 2.6.1, Platform Windows-10-10.0.17763-SP0
  1154. 2019-06-22 11:36:27 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'SportsSpider', 'LOG_FILE': 'SportsSpiderlog20190622.txt', 'LOG_LEVEL': 'INFO', 'NEWSPIDER_MODULE': 'collectSports.spiders', 'SPIDER_MODULES': ['collectSports.spiders'], 'USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 UBrowser/6.2.4098.3 Safari/537.36'}
  1155. 2019-06-22 11:36:27 [scrapy.extensions.telnet] INFO: Telnet Password: 35a021e2bc32beed
  1156. 2019-06-22 11:36:27 [scrapy.middleware] INFO: Enabled extensions:
  1157. ['scrapy.extensions.corestats.CoreStats',
  1158. 'scrapy.extensions.telnet.TelnetConsole',
  1159. 'scrapy.extensions.logstats.LogStats']
  1160. 2019-06-22 11:36:27 [scrapy.middleware] INFO: Enabled downloader middlewares:
  1161. ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
  1162. 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
  1163. 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
  1164. 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
  1165. 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
  1166. 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
  1167. 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
  1168. 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
  1169. 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
  1170. 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
  1171. 'scrapy.downloadermiddlewares.stats.DownloaderStats']
  1172. 2019-06-22 11:36:27 [scrapy.middleware] INFO: Enabled spider middlewares:
  1173. ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
  1174. 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
  1175. 'scrapy.spidermiddlewares.referer.RefererMiddleware',
  1176. 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
  1177. 'scrapy.spidermiddlewares.depth.DepthMiddleware']
  1178. 2019-06-22 11:36:27 [scrapy.middleware] INFO: Enabled item pipelines:
  1179. ['collectSports.pipelines.sportslst.SportslstPipeline']
  1180. 2019-06-22 11:36:27 [scrapy.core.engine] INFO: Spider opened
  1181. 2019-06-22 11:36:27 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
  1182. 2019-06-22 11:36:27 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
  1183. 2019-06-22 11:36:28 [scrapy.core.downloader.tls] WARNING: Ignoring error while verifying certificate from host "205.201.4.177" (exception: ValueError('Invalid DNS-ID.'))
  1184. 2019-06-22 11:38:03 [scrapy.extensions.logstats] INFO: Crawled 1 pages (at 1 pages/min), scraped 1 items (at 1 items/min)
  1185. 2019-06-22 11:38:03 [scrapy.core.engine] INFO: Closing spider (finished)
  1186. 2019-06-22 11:38:03 [scrapy.core.engine] ERROR: Scraper close failure
  1187. Traceback (most recent call last):
  1188. File "C:\ProgramData\Anaconda3\lib\site-packages\twisted\internet\defer.py", line 654, in _runCallbacks
  1189. current.result = callback(current.result, *args, **kw)
  1190. File "D:\workspace\sports_collect\collectSports\pipelines\sportslst.py", line 170, in close_spider
  1191. self.client.close()
  1192. AttributeError: 'SportslstPipeline' object has no attribute 'client'
  1193. 2019-06-22 11:38:03 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
  1194. {'downloader/request_bytes': 455,
  1195. 'downloader/request_count': 1,
  1196. 'downloader/request_method_count/GET': 1,
  1197. 'downloader/response_bytes': 4730,
  1198. 'downloader/response_count': 1,
  1199. 'downloader/response_status_count/200': 1,
  1200. 'finish_reason': 'finished',
  1201. 'finish_time': datetime.datetime(2019, 6, 22, 3, 38, 3, 318581),
  1202. 'item_scraped_count': 1,
  1203. 'log_count/ERROR': 1,
  1204. 'log_count/INFO': 10,
  1205. 'log_count/WARNING': 1,
  1206. 'response_received_count': 1,
  1207. 'scheduler/dequeued': 1,
  1208. 'scheduler/dequeued/memory': 1,
  1209. 'scheduler/enqueued': 1,
  1210. 'scheduler/enqueued/memory': 1,
  1211. 'start_time': datetime.datetime(2019, 6, 22, 3, 36, 27, 790549)}
  1212. 2019-06-22 11:38:03 [scrapy.core.engine] INFO: Spider closed (finished)
  1213. 2019-06-22 11:39:28 [scrapy.utils.log] INFO: Scrapy 1.6.0 started (bot: SportsSpider)
  1214. 2019-06-22 11:39:28 [scrapy.utils.log] INFO: Versions: lxml 4.3.2.0, libxml2 2.9.9, cssselect 1.0.3, parsel 1.5.1, w3lib 1.20.0, Twisted 19.2.1, Python 3.7.3 (default, Mar 27 2019, 17:13:21) [MSC v.1915 64 bit (AMD64)], pyOpenSSL 19.0.0 (OpenSSL 1.1.1b 26 Feb 2019), cryptography 2.6.1, Platform Windows-10-10.0.17763-SP0
  1215. 2019-06-22 11:39:28 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'SportsSpider', 'LOG_FILE': 'SportsSpiderlog20190622.txt', 'LOG_LEVEL': 'INFO', 'NEWSPIDER_MODULE': 'collectSports.spiders', 'SPIDER_MODULES': ['collectSports.spiders'], 'USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 UBrowser/6.2.4098.3 Safari/537.36'}
  1216. 2019-06-22 11:39:28 [scrapy.extensions.telnet] INFO: Telnet Password: 1bd2fb0ec3dc9450
  1217. 2019-06-22 11:39:28 [scrapy.middleware] INFO: Enabled extensions:
  1218. ['scrapy.extensions.corestats.CoreStats',
  1219. 'scrapy.extensions.telnet.TelnetConsole',
  1220. 'scrapy.extensions.logstats.LogStats']
  1221. 2019-06-22 11:39:29 [scrapy.middleware] INFO: Enabled downloader middlewares:
  1222. ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
  1223. 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
  1224. 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
  1225. 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
  1226. 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
  1227. 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
  1228. 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
  1229. 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
  1230. 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
  1231. 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
  1232. 'scrapy.downloadermiddlewares.stats.DownloaderStats']
  1233. 2019-06-22 11:39:29 [scrapy.middleware] INFO: Enabled spider middlewares:
  1234. ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
  1235. 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
  1236. 'scrapy.spidermiddlewares.referer.RefererMiddleware',
  1237. 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
  1238. 'scrapy.spidermiddlewares.depth.DepthMiddleware']
  1239. 2019-06-22 11:39:29 [scrapy.middleware] INFO: Enabled item pipelines:
  1240. ['collectSports.pipelines.sportslst.SportslstPipeline']
  1241. 2019-06-22 11:39:29 [scrapy.core.engine] INFO: Spider opened
  1242. 2019-06-22 11:39:29 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
  1243. 2019-06-22 11:39:29 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
  1244. 2019-06-22 11:39:29 [scrapy.core.downloader.tls] WARNING: Ignoring error while verifying certificate from host "205.201.4.177" (exception: ValueError('Invalid DNS-ID.'))
  1245. 2019-06-22 11:40:22 [scrapy.utils.log] INFO: Scrapy 1.6.0 started (bot: SportsSpider)
  1246. 2019-06-22 11:40:22 [scrapy.utils.log] INFO: Versions: lxml 4.3.2.0, libxml2 2.9.9, cssselect 1.0.3, parsel 1.5.1, w3lib 1.20.0, Twisted 19.2.1, Python 3.7.3 (default, Mar 27 2019, 17:13:21) [MSC v.1915 64 bit (AMD64)], pyOpenSSL 19.0.0 (OpenSSL 1.1.1b 26 Feb 2019), cryptography 2.6.1, Platform Windows-10-10.0.17763-SP0
  1247. 2019-06-22 11:40:22 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'SportsSpider', 'LOG_FILE': 'SportsSpiderlog20190622.txt', 'LOG_LEVEL': 'INFO', 'NEWSPIDER_MODULE': 'collectSports.spiders', 'SPIDER_MODULES': ['collectSports.spiders'], 'USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 UBrowser/6.2.4098.3 Safari/537.36'}
  1248. 2019-06-22 11:40:22 [scrapy.extensions.telnet] INFO: Telnet Password: 551a6dcf05de3a37
  1249. 2019-06-22 11:40:22 [scrapy.middleware] INFO: Enabled extensions:
  1250. ['scrapy.extensions.corestats.CoreStats',
  1251. 'scrapy.extensions.telnet.TelnetConsole',
  1252. 'scrapy.extensions.logstats.LogStats']
  1253. 2019-06-22 11:40:22 [scrapy.middleware] INFO: Enabled downloader middlewares:
  1254. ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
  1255. 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
  1256. 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
  1257. 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
  1258. 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
  1259. 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
  1260. 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
  1261. 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
  1262. 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
  1263. 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
  1264. 'scrapy.downloadermiddlewares.stats.DownloaderStats']
  1265. 2019-06-22 11:40:22 [scrapy.middleware] INFO: Enabled spider middlewares:
  1266. ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
  1267. 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
  1268. 'scrapy.spidermiddlewares.referer.RefererMiddleware',
  1269. 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
  1270. 'scrapy.spidermiddlewares.depth.DepthMiddleware']
  1271. 2019-06-22 11:40:23 [scrapy.middleware] INFO: Enabled item pipelines:
  1272. ['collectSports.pipelines.sportslst.SportslstPipeline']
  1273. 2019-06-22 11:40:23 [scrapy.core.engine] INFO: Spider opened
  1274. 2019-06-22 11:40:23 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
  1275. 2019-06-22 11:40:23 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
  1276. 2019-06-22 11:40:23 [scrapy.core.downloader.tls] WARNING: Ignoring error while verifying certificate from host "205.201.4.177" (exception: ValueError('Invalid DNS-ID.'))
  1277. 2019-06-22 11:41:46 [scrapy.extensions.logstats] INFO: Crawled 1 pages (at 1 pages/min), scraped 1 items (at 1 items/min)
  1278. 2019-06-22 11:41:46 [scrapy.core.engine] INFO: Closing spider (finished)
  1279. 2019-06-22 11:41:46 [scrapy.core.engine] ERROR: Scraper close failure
  1280. Traceback (most recent call last):
  1281. File "C:\ProgramData\Anaconda3\lib\site-packages\twisted\internet\defer.py", line 654, in _runCallbacks
  1282. current.result = callback(current.result, *args, **kw)
  1283. File "D:\workspace\sports_collect\collectSports\pipelines\sportslst.py", line 170, in close_spider
  1284. self.client.close()
  1285. AttributeError: 'SportslstPipeline' object has no attribute 'client'
  1286. 2019-06-22 11:41:46 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
  1287. {'downloader/request_bytes': 490,
  1288. 'downloader/request_count': 1,
  1289. 'downloader/request_method_count/GET': 1,
  1290. 'downloader/response_bytes': 19800,
  1291. 'downloader/response_count': 1,
  1292. 'downloader/response_status_count/200': 1,
  1293. 'finish_reason': 'finished',
  1294. 'finish_time': datetime.datetime(2019, 6, 22, 3, 41, 46, 856780),
  1295. 'item_scraped_count': 1,
  1296. 'log_count/ERROR': 1,
  1297. 'log_count/INFO': 10,
  1298. 'log_count/WARNING': 1,
  1299. 'response_received_count': 1,
  1300. 'scheduler/dequeued': 1,
  1301. 'scheduler/dequeued/memory': 1,
  1302. 'scheduler/enqueued': 1,
  1303. 'scheduler/enqueued/memory': 1,
  1304. 'start_time': datetime.datetime(2019, 6, 22, 3, 40, 23, 8292)}
  1305. 2019-06-22 11:41:46 [scrapy.core.engine] INFO: Spider closed (finished)