| 123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432433434435436437438439440441442443444445446447448449450451452453454455456457458459460461462463464465466467468469470471472473474475476477478479480481482483484485486487488489490491492493494495496497498499500501502503504505506507508509510511512513514515516517518519520521522523524525526527528529530531532533534535536537538539540541542543544545546547548549550551552553554555556557558559560561562563564565566567568569570571572573574575576577578579580581582583584585586587588589590591592593594595596597598599600601602603604605606607608609610611612613614615616617618619620621622623624625626627628629630631632633634635636637638639640641642643644645646647648649650651652653654655656657658659660661662663664665666667668669670671672673674675676677678679680681682683684685686687688689690691692693694695696697698699700701702703704705706707708709710711712713714715716717718719720721722723724725726727728729730731732733734735736737738739740741742743744745746747748749750751752753754755756757758759760761762763764765766767768769770771772773774775776777778779780781782783784785786787788789790791792793794795796797798799800801802803804805806807808809810811812813814815816817818819820821822823824825826827828829830831832833834835836837838839840841842843844845846847848849850851852853854855856857858859860861862863864865866867868869870871872873874875876877878879880881882883884885886887888889890891892893894895896897898899900901902903904905906907908909910911912913914915916917918919920921922923924925926927928929930931932933934935936937938939940941942943944945946947948949950951952953954955956957958959960961962963964965966967968969970971972973974975976977978979980981982983984985986987988989990991992993994995996997998999100010011002100310041005100610071008100910101011101210131014101510161017101810191020102110221023102410251026102710281029103010311032103310341035103610371038103910401041104210431044104510461047104810491050105110521053105410551056105710581059106010611062106310641065106610671068106910701071107210731074107510761077107810791080108110821083108410851086108710881089109010911092109310941095109610971098109911001101110211031104110511061107110811091110111111121113111411151116111711181119112011211122112311241125112611271128112911301131113211331134113511361137113811391140114111421143114411451146114711481149115011511152115311541155115611571158115911601161116211631164116511661167116811691170117111721173117411751176117711781179118011811182118311841185118611871188118911901191119211931194119511961197119811991200120112021203120412051206120712081209121012111212121312141215121612171218121912201221122212231224122512261227122812291230123112321233123412351236123712381239124012411242124312441245124612471248124912501251125212531254125512561257125812591260126112621263126412651266126712681269127012711272127312741275127612771278127912801281128212831284128512861287128812891290129112921293129412951296129712981299130013011302130313041305 |
- 2019-06-22 09:08:13 [scrapy.utils.log] INFO: Scrapy 1.6.0 started (bot: SportsSpider)
- 2019-06-22 09:08:13 [scrapy.utils.log] INFO: Versions: lxml 4.3.2.0, libxml2 2.9.9, cssselect 1.0.3, parsel 1.5.1, w3lib 1.20.0, Twisted 19.2.1, Python 3.7.3 (default, Mar 27 2019, 17:13:21) [MSC v.1915 64 bit (AMD64)], pyOpenSSL 19.0.0 (OpenSSL 1.1.1b 26 Feb 2019), cryptography 2.6.1, Platform Windows-10-10.0.17763-SP0
- 2019-06-22 09:08:13 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'SportsSpider', 'LOG_FILE': 'SportsSpiderlog20190622.txt', 'LOG_LEVEL': 'INFO', 'NEWSPIDER_MODULE': 'collectSports.spiders', 'SPIDER_MODULES': ['collectSports.spiders'], 'USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 UBrowser/6.2.4098.3 Safari/537.36'}
- 2019-06-22 09:08:13 [scrapy.extensions.telnet] INFO: Telnet Password: 66adf74d240537aa
- 2019-06-22 09:08:13 [scrapy.middleware] INFO: Enabled extensions:
- ['scrapy.extensions.corestats.CoreStats',
- 'scrapy.extensions.telnet.TelnetConsole',
- 'scrapy.extensions.logstats.LogStats']
- 2019-06-22 09:08:13 [scrapy.middleware] INFO: Enabled downloader middlewares:
- ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
- 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
- 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
- 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
- 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
- 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
- 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
- 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
- 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
- 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
- 'scrapy.downloadermiddlewares.stats.DownloaderStats']
- 2019-06-22 09:08:13 [scrapy.middleware] INFO: Enabled spider middlewares:
- ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
- 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
- 'scrapy.spidermiddlewares.referer.RefererMiddleware',
- 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
- 'scrapy.spidermiddlewares.depth.DepthMiddleware']
- 2019-06-22 09:08:14 [scrapy.middleware] INFO: Enabled item pipelines:
- ['collectSports.pipelines.sportslst.SportslstPipeline']
- 2019-06-22 09:08:14 [scrapy.core.engine] INFO: Spider opened
- 2019-06-22 09:08:14 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
- 2019-06-22 09:08:14 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
- 2019-06-22 09:08:19 [scrapy.core.downloader.tls] WARNING: Ignoring error while verifying certificate from host "199.26.100.178" (exception: ValueError('Invalid DNS-ID.'))
- 2019-06-22 09:09:53 [scrapy.core.scraper] ERROR: Spider error processing <GET https://199.26.100.178/app/member/FT_browse/body_var.php?uid=opqygihvm21680472l14201&rtype=r&langx=zh-cn&mtype=4&page_no=1&league_id=&hot_game=&isie11=27N%27> (referer: None)
- Traceback (most recent call last):
- File "C:\ProgramData\Anaconda3\lib\site-packages\scrapy\utils\defer.py", line 102, in iter_errback
- yield next(it)
- File "C:\ProgramData\Anaconda3\lib\site-packages\scrapy\spidermiddlewares\offsite.py", line 29, in process_spider_output
- for x in result:
- File "C:\ProgramData\Anaconda3\lib\site-packages\scrapy\spidermiddlewares\referer.py", line 339, in <genexpr>
- return (_set_referer(r) for r in result or ())
- File "C:\ProgramData\Anaconda3\lib\site-packages\scrapy\spidermiddlewares\urllength.py", line 37, in <genexpr>
- return (r for r in result or () if _filter(r))
- File "C:\ProgramData\Anaconda3\lib\site-packages\scrapy\spidermiddlewares\depth.py", line 58, in <genexpr>
- return (r for r in result or () if _filter(r))
- File "D:\workspace\sports_collect\collectSports\spiders\sportslst.py", line 58, in parse
- re = cbk(response.body)
- File "D:\workspace\sports_collect\collectSports\mcollect\hg0088\Resolver.py", line 20, in today
- re = self.mixbodyv(data, 'today', '/html/head/script[2]/text()')
- File "D:\workspace\sports_collect\collectSports\mcollect\hg0088\Resolver.py", line 35, in mixbodyv
- text = ls[0]
- IndexError: list index out of range
- 2019-06-22 09:09:53 [scrapy.extensions.logstats] INFO: Crawled 1 pages (at 1 pages/min), scraped 0 items (at 0 items/min)
- 2019-06-22 09:09:53 [scrapy.core.engine] INFO: Closing spider (finished)
- 2019-06-22 09:09:53 [scrapy.core.engine] ERROR: Scraper close failure
- Traceback (most recent call last):
- File "C:\ProgramData\Anaconda3\lib\site-packages\twisted\internet\defer.py", line 654, in _runCallbacks
- current.result = callback(current.result, *args, **kw)
- File "D:\workspace\sports_collect\collectSports\pipelines\sportslst.py", line 167, in close_spider
- self.client.close()
- AttributeError: 'SportslstPipeline' object has no attribute 'client'
- 2019-06-22 09:09:53 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
- {'downloader/request_bytes': 442,
- 'downloader/request_count': 1,
- 'downloader/request_method_count/GET': 1,
- 'downloader/response_bytes': 306,
- 'downloader/response_count': 1,
- 'downloader/response_status_count/200': 1,
- 'finish_reason': 'finished',
- 'finish_time': datetime.datetime(2019, 6, 22, 1, 9, 53, 348330),
- 'log_count/ERROR': 2,
- 'log_count/INFO': 10,
- 'log_count/WARNING': 1,
- 'response_received_count': 1,
- 'scheduler/dequeued': 1,
- 'scheduler/dequeued/memory': 1,
- 'scheduler/enqueued': 1,
- 'scheduler/enqueued/memory': 1,
- 'spider_exceptions/IndexError': 1,
- 'start_time': datetime.datetime(2019, 6, 22, 1, 8, 14, 84727)}
- 2019-06-22 09:09:53 [scrapy.core.engine] INFO: Spider closed (finished)
- 2019-06-22 09:10:02 [scrapy.utils.log] INFO: Scrapy 1.6.0 started (bot: SportsSpider)
- 2019-06-22 09:10:02 [scrapy.utils.log] INFO: Versions: lxml 4.3.2.0, libxml2 2.9.9, cssselect 1.0.3, parsel 1.5.1, w3lib 1.20.0, Twisted 19.2.1, Python 3.7.3 (default, Mar 27 2019, 17:13:21) [MSC v.1915 64 bit (AMD64)], pyOpenSSL 19.0.0 (OpenSSL 1.1.1b 26 Feb 2019), cryptography 2.6.1, Platform Windows-10-10.0.17763-SP0
- 2019-06-22 09:10:02 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'SportsSpider', 'LOG_FILE': 'SportsSpiderlog20190622.txt', 'LOG_LEVEL': 'INFO', 'NEWSPIDER_MODULE': 'collectSports.spiders', 'SPIDER_MODULES': ['collectSports.spiders'], 'USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 UBrowser/6.2.4098.3 Safari/537.36'}
- 2019-06-22 09:10:02 [scrapy.extensions.telnet] INFO: Telnet Password: c5dee860fe1a6f70
- 2019-06-22 09:10:02 [scrapy.middleware] INFO: Enabled extensions:
- ['scrapy.extensions.corestats.CoreStats',
- 'scrapy.extensions.telnet.TelnetConsole',
- 'scrapy.extensions.logstats.LogStats']
- 2019-06-22 09:10:02 [scrapy.middleware] INFO: Enabled downloader middlewares:
- ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
- 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
- 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
- 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
- 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
- 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
- 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
- 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
- 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
- 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
- 'scrapy.downloadermiddlewares.stats.DownloaderStats']
- 2019-06-22 09:10:02 [scrapy.middleware] INFO: Enabled spider middlewares:
- ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
- 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
- 'scrapy.spidermiddlewares.referer.RefererMiddleware',
- 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
- 'scrapy.spidermiddlewares.depth.DepthMiddleware']
- 2019-06-22 09:10:02 [scrapy.middleware] INFO: Enabled item pipelines:
- ['collectSports.pipelines.sportslst.SportslstPipeline']
- 2019-06-22 09:10:02 [scrapy.core.engine] INFO: Spider opened
- 2019-06-22 09:10:02 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
- 2019-06-22 09:10:02 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
- 2019-06-22 09:10:02 [scrapy.core.downloader.tls] WARNING: Ignoring error while verifying certificate from host "199.26.100.178" (exception: ValueError('Invalid DNS-ID.'))
- 2019-06-22 09:10:07 [scrapy.core.scraper] ERROR: Spider error processing <GET https://199.26.100.178/app/member/FT_browse/body_var.php?uid=opqygihvm21680472l14201&rtype=r&langx=zh-cn&mtype=4&page_no=1&league_id=&hot_game=&isie11=27N%27> (referer: None)
- Traceback (most recent call last):
- File "C:\ProgramData\Anaconda3\lib\site-packages\scrapy\utils\defer.py", line 102, in iter_errback
- yield next(it)
- File "C:\ProgramData\Anaconda3\lib\site-packages\scrapy\spidermiddlewares\offsite.py", line 29, in process_spider_output
- for x in result:
- File "C:\ProgramData\Anaconda3\lib\site-packages\scrapy\spidermiddlewares\referer.py", line 339, in <genexpr>
- return (_set_referer(r) for r in result or ())
- File "C:\ProgramData\Anaconda3\lib\site-packages\scrapy\spidermiddlewares\urllength.py", line 37, in <genexpr>
- return (r for r in result or () if _filter(r))
- File "C:\ProgramData\Anaconda3\lib\site-packages\scrapy\spidermiddlewares\depth.py", line 58, in <genexpr>
- return (r for r in result or () if _filter(r))
- File "D:\workspace\sports_collect\collectSports\spiders\sportslst.py", line 58, in parse
- re = cbk(response.body)
- File "D:\workspace\sports_collect\collectSports\mcollect\hg0088\Resolver.py", line 20, in today
- re = self.mixbodyv(data, 'today', '/html/head/script[2]/text()')
- File "D:\workspace\sports_collect\collectSports\mcollect\hg0088\Resolver.py", line 35, in mixbodyv
- text = ls[0]
- IndexError: list index out of range
- 2019-06-22 09:10:07 [scrapy.core.engine] INFO: Closing spider (finished)
- 2019-06-22 09:10:07 [scrapy.core.engine] ERROR: Scraper close failure
- Traceback (most recent call last):
- File "C:\ProgramData\Anaconda3\lib\site-packages\twisted\internet\defer.py", line 654, in _runCallbacks
- current.result = callback(current.result, *args, **kw)
- File "D:\workspace\sports_collect\collectSports\pipelines\sportslst.py", line 167, in close_spider
- self.client.close()
- AttributeError: 'SportslstPipeline' object has no attribute 'client'
- 2019-06-22 09:10:07 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
- {'downloader/request_bytes': 442,
- 'downloader/request_count': 1,
- 'downloader/request_method_count/GET': 1,
- 'downloader/response_bytes': 306,
- 'downloader/response_count': 1,
- 'downloader/response_status_count/200': 1,
- 'finish_reason': 'finished',
- 'finish_time': datetime.datetime(2019, 6, 22, 1, 10, 7, 465580),
- 'log_count/ERROR': 2,
- 'log_count/INFO': 9,
- 'log_count/WARNING': 1,
- 'response_received_count': 1,
- 'scheduler/dequeued': 1,
- 'scheduler/dequeued/memory': 1,
- 'scheduler/enqueued': 1,
- 'scheduler/enqueued/memory': 1,
- 'spider_exceptions/IndexError': 1,
- 'start_time': datetime.datetime(2019, 6, 22, 1, 10, 2, 470597)}
- 2019-06-22 09:10:07 [scrapy.core.engine] INFO: Spider closed (finished)
- 2019-06-22 09:10:14 [scrapy.utils.log] INFO: Scrapy 1.6.0 started (bot: SportsSpider)
- 2019-06-22 09:10:14 [scrapy.utils.log] INFO: Versions: lxml 4.3.2.0, libxml2 2.9.9, cssselect 1.0.3, parsel 1.5.1, w3lib 1.20.0, Twisted 19.2.1, Python 3.7.3 (default, Mar 27 2019, 17:13:21) [MSC v.1915 64 bit (AMD64)], pyOpenSSL 19.0.0 (OpenSSL 1.1.1b 26 Feb 2019), cryptography 2.6.1, Platform Windows-10-10.0.17763-SP0
- 2019-06-22 09:10:14 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'SportsSpider', 'LOG_FILE': 'SportsSpiderlog20190622.txt', 'LOG_LEVEL': 'INFO', 'NEWSPIDER_MODULE': 'collectSports.spiders', 'SPIDER_MODULES': ['collectSports.spiders'], 'USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 UBrowser/6.2.4098.3 Safari/537.36'}
- 2019-06-22 09:10:14 [scrapy.extensions.telnet] INFO: Telnet Password: fbbf17512304b0d7
- 2019-06-22 09:10:14 [scrapy.middleware] INFO: Enabled extensions:
- ['scrapy.extensions.corestats.CoreStats',
- 'scrapy.extensions.telnet.TelnetConsole',
- 'scrapy.extensions.logstats.LogStats']
- 2019-06-22 09:10:14 [scrapy.middleware] INFO: Enabled downloader middlewares:
- ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
- 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
- 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
- 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
- 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
- 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
- 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
- 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
- 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
- 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
- 'scrapy.downloadermiddlewares.stats.DownloaderStats']
- 2019-06-22 09:10:14 [scrapy.middleware] INFO: Enabled spider middlewares:
- ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
- 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
- 'scrapy.spidermiddlewares.referer.RefererMiddleware',
- 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
- 'scrapy.spidermiddlewares.depth.DepthMiddleware']
- 2019-06-22 09:10:14 [scrapy.middleware] INFO: Enabled item pipelines:
- ['collectSports.pipelines.sportslst.SportslstPipeline']
- 2019-06-22 09:10:14 [scrapy.core.engine] INFO: Spider opened
- 2019-06-22 09:10:14 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
- 2019-06-22 09:10:14 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
- 2019-06-22 09:10:14 [scrapy.core.downloader.tls] WARNING: Ignoring error while verifying certificate from host "199.26.100.178" (exception: ValueError('Invalid DNS-ID.'))
- 2019-06-22 09:44:50 [scrapy.core.scraper] ERROR: Spider error processing <GET https://199.26.100.178/app/member/FT_browse/body_var.php?uid=opqygihvm21680472l14201&rtype=r&langx=zh-cn&mtype=4&page_no=1&league_id=&hot_game=&isie11=27N%27> (referer: None)
- Traceback (most recent call last):
- File "C:\ProgramData\Anaconda3\lib\site-packages\scrapy\utils\defer.py", line 102, in iter_errback
- yield next(it)
- File "C:\ProgramData\Anaconda3\lib\site-packages\scrapy\spidermiddlewares\offsite.py", line 29, in process_spider_output
- for x in result:
- File "C:\ProgramData\Anaconda3\lib\site-packages\scrapy\spidermiddlewares\referer.py", line 339, in <genexpr>
- return (_set_referer(r) for r in result or ())
- File "C:\ProgramData\Anaconda3\lib\site-packages\scrapy\spidermiddlewares\urllength.py", line 37, in <genexpr>
- return (r for r in result or () if _filter(r))
- File "C:\ProgramData\Anaconda3\lib\site-packages\scrapy\spidermiddlewares\depth.py", line 58, in <genexpr>
- return (r for r in result or () if _filter(r))
- File "D:\workspace\sports_collect\collectSports\spiders\sportslst.py", line 58, in parse
- re = cbk(response.body)
- File "D:\workspace\sports_collect\collectSports\mcollect\hg0088\Resolver.py", line 20, in today
- # // *[ @ id = "title_today"]
- File "D:\workspace\sports_collect\collectSports\mcollect\hg0088\Resolver.py", line 35, in mixbodyv
- ls = doc.xpath(content_xpath)
- IndexError: list index out of range
- 2019-06-22 09:44:50 [scrapy.extensions.logstats] INFO: Crawled 1 pages (at 1 pages/min), scraped 0 items (at 0 items/min)
- 2019-06-22 09:44:50 [scrapy.core.engine] INFO: Closing spider (finished)
- 2019-06-22 09:44:50 [scrapy.core.engine] ERROR: Scraper close failure
- Traceback (most recent call last):
- File "C:\ProgramData\Anaconda3\lib\site-packages\twisted\internet\defer.py", line 654, in _runCallbacks
- current.result = callback(current.result, *args, **kw)
- File "D:\workspace\sports_collect\collectSports\pipelines\sportslst.py", line 167, in close_spider
- self.client.close()
- AttributeError: 'SportslstPipeline' object has no attribute 'client'
- 2019-06-22 09:44:50 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
- {'downloader/request_bytes': 442,
- 'downloader/request_count': 1,
- 'downloader/request_method_count/GET': 1,
- 'downloader/response_bytes': 306,
- 'downloader/response_count': 1,
- 'downloader/response_status_count/200': 1,
- 'finish_reason': 'finished',
- 'finish_time': datetime.datetime(2019, 6, 22, 1, 44, 50, 513810),
- 'log_count/ERROR': 2,
- 'log_count/INFO': 10,
- 'log_count/WARNING': 1,
- 'response_received_count': 1,
- 'scheduler/dequeued': 1,
- 'scheduler/dequeued/memory': 1,
- 'scheduler/enqueued': 1,
- 'scheduler/enqueued/memory': 1,
- 'spider_exceptions/IndexError': 1,
- 'start_time': datetime.datetime(2019, 6, 22, 1, 10, 14, 577435)}
- 2019-06-22 09:44:50 [scrapy.core.engine] INFO: Spider closed (finished)
- 2019-06-22 09:45:00 [scrapy.utils.log] INFO: Scrapy 1.6.0 started (bot: SportsSpider)
- 2019-06-22 09:45:00 [scrapy.utils.log] INFO: Versions: lxml 4.3.2.0, libxml2 2.9.9, cssselect 1.0.3, parsel 1.5.1, w3lib 1.20.0, Twisted 19.2.1, Python 3.7.3 (default, Mar 27 2019, 17:13:21) [MSC v.1915 64 bit (AMD64)], pyOpenSSL 19.0.0 (OpenSSL 1.1.1b 26 Feb 2019), cryptography 2.6.1, Platform Windows-10-10.0.17763-SP0
- 2019-06-22 09:45:00 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'SportsSpider', 'LOG_FILE': 'SportsSpiderlog20190622.txt', 'LOG_LEVEL': 'INFO', 'NEWSPIDER_MODULE': 'collectSports.spiders', 'SPIDER_MODULES': ['collectSports.spiders'], 'USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 UBrowser/6.2.4098.3 Safari/537.36'}
- 2019-06-22 09:45:00 [scrapy.extensions.telnet] INFO: Telnet Password: bf58da38955895de
- 2019-06-22 09:45:00 [scrapy.middleware] INFO: Enabled extensions:
- ['scrapy.extensions.corestats.CoreStats',
- 'scrapy.extensions.telnet.TelnetConsole',
- 'scrapy.extensions.logstats.LogStats']
- 2019-06-22 09:45:00 [scrapy.middleware] INFO: Enabled downloader middlewares:
- ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
- 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
- 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
- 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
- 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
- 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
- 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
- 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
- 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
- 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
- 'scrapy.downloadermiddlewares.stats.DownloaderStats']
- 2019-06-22 09:45:00 [scrapy.middleware] INFO: Enabled spider middlewares:
- ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
- 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
- 'scrapy.spidermiddlewares.referer.RefererMiddleware',
- 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
- 'scrapy.spidermiddlewares.depth.DepthMiddleware']
- 2019-06-22 09:45:00 [scrapy.middleware] INFO: Enabled item pipelines:
- ['collectSports.pipelines.sportslst.SportslstPipeline']
- 2019-06-22 09:45:00 [scrapy.core.engine] INFO: Spider opened
- 2019-06-22 09:45:00 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
- 2019-06-22 09:45:00 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
- 2019-06-22 09:45:00 [scrapy.core.downloader.tls] WARNING: Ignoring error while verifying certificate from host "199.26.100.178" (exception: ValueError('Invalid DNS-ID.'))
- 2019-06-22 10:08:57 [scrapy.utils.log] INFO: Scrapy 1.6.0 started (bot: SportsSpider)
- 2019-06-22 10:08:57 [scrapy.utils.log] INFO: Versions: lxml 4.3.2.0, libxml2 2.9.9, cssselect 1.0.3, parsel 1.5.1, w3lib 1.20.0, Twisted 19.2.1, Python 3.7.3 (default, Mar 27 2019, 17:13:21) [MSC v.1915 64 bit (AMD64)], pyOpenSSL 19.0.0 (OpenSSL 1.1.1b 26 Feb 2019), cryptography 2.6.1, Platform Windows-10-10.0.17763-SP0
- 2019-06-22 10:08:57 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'SportsSpider', 'LOG_FILE': 'SportsSpiderlog20190622.txt', 'LOG_LEVEL': 'INFO', 'NEWSPIDER_MODULE': 'collectSports.spiders', 'SPIDER_MODULES': ['collectSports.spiders'], 'USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 UBrowser/6.2.4098.3 Safari/537.36'}
- 2019-06-22 10:08:57 [scrapy.extensions.telnet] INFO: Telnet Password: 85f4a4c56defc508
- 2019-06-22 10:08:57 [scrapy.middleware] INFO: Enabled extensions:
- ['scrapy.extensions.corestats.CoreStats',
- 'scrapy.extensions.telnet.TelnetConsole',
- 'scrapy.extensions.logstats.LogStats']
- 2019-06-22 10:08:57 [scrapy.middleware] INFO: Enabled downloader middlewares:
- ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
- 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
- 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
- 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
- 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
- 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
- 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
- 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
- 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
- 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
- 'scrapy.downloadermiddlewares.stats.DownloaderStats']
- 2019-06-22 10:08:57 [scrapy.middleware] INFO: Enabled spider middlewares:
- ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
- 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
- 'scrapy.spidermiddlewares.referer.RefererMiddleware',
- 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
- 'scrapy.spidermiddlewares.depth.DepthMiddleware']
- 2019-06-22 10:08:57 [scrapy.middleware] INFO: Enabled item pipelines:
- ['collectSports.pipelines.sportslst.SportslstPipeline']
- 2019-06-22 10:08:57 [scrapy.core.engine] INFO: Spider opened
- 2019-06-22 10:08:57 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
- 2019-06-22 10:08:57 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
- 2019-06-22 10:08:58 [scrapy.core.downloader.tls] WARNING: Ignoring error while verifying certificate from host "205.201.4.177" (exception: ValueError('Invalid DNS-ID.'))
- 2019-06-22 10:18:00 [scrapy.utils.log] INFO: Scrapy 1.6.0 started (bot: SportsSpider)
- 2019-06-22 10:18:00 [scrapy.utils.log] INFO: Versions: lxml 4.3.2.0, libxml2 2.9.9, cssselect 1.0.3, parsel 1.5.1, w3lib 1.20.0, Twisted 19.2.1, Python 3.7.3 (default, Mar 27 2019, 17:13:21) [MSC v.1915 64 bit (AMD64)], pyOpenSSL 19.0.0 (OpenSSL 1.1.1b 26 Feb 2019), cryptography 2.6.1, Platform Windows-10-10.0.17763-SP0
- 2019-06-22 10:18:00 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'SportsSpider', 'LOG_FILE': 'SportsSpiderlog20190622.txt', 'LOG_LEVEL': 'INFO', 'NEWSPIDER_MODULE': 'collectSports.spiders', 'SPIDER_MODULES': ['collectSports.spiders'], 'USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 UBrowser/6.2.4098.3 Safari/537.36'}
- 2019-06-22 10:18:00 [scrapy.extensions.telnet] INFO: Telnet Password: b71628a0b76cdb69
- 2019-06-22 10:18:00 [scrapy.middleware] INFO: Enabled extensions:
- ['scrapy.extensions.corestats.CoreStats',
- 'scrapy.extensions.telnet.TelnetConsole',
- 'scrapy.extensions.logstats.LogStats']
- 2019-06-22 10:18:00 [scrapy.middleware] INFO: Enabled downloader middlewares:
- ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
- 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
- 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
- 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
- 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
- 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
- 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
- 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
- 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
- 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
- 'scrapy.downloadermiddlewares.stats.DownloaderStats']
- 2019-06-22 10:18:00 [scrapy.middleware] INFO: Enabled spider middlewares:
- ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
- 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
- 'scrapy.spidermiddlewares.referer.RefererMiddleware',
- 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
- 'scrapy.spidermiddlewares.depth.DepthMiddleware']
- 2019-06-22 10:18:00 [scrapy.middleware] INFO: Enabled item pipelines:
- ['collectSports.pipelines.sportslst.SportslstPipeline']
- 2019-06-22 10:18:00 [scrapy.core.engine] INFO: Spider opened
- 2019-06-22 10:18:00 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
- 2019-06-22 10:18:00 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
- 2019-06-22 10:18:00 [scrapy.core.downloader.tls] WARNING: Ignoring error while verifying certificate from host "205.201.4.177" (exception: ValueError('Invalid DNS-ID.'))
- 2019-06-22 10:19:24 [scrapy.utils.log] INFO: Scrapy 1.6.0 started (bot: SportsSpider)
- 2019-06-22 10:19:24 [scrapy.utils.log] INFO: Versions: lxml 4.3.2.0, libxml2 2.9.9, cssselect 1.0.3, parsel 1.5.1, w3lib 1.20.0, Twisted 19.2.1, Python 3.7.3 (default, Mar 27 2019, 17:13:21) [MSC v.1915 64 bit (AMD64)], pyOpenSSL 19.0.0 (OpenSSL 1.1.1b 26 Feb 2019), cryptography 2.6.1, Platform Windows-10-10.0.17763-SP0
- 2019-06-22 10:19:24 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'SportsSpider', 'LOG_FILE': 'SportsSpiderlog20190622.txt', 'LOG_LEVEL': 'INFO', 'NEWSPIDER_MODULE': 'collectSports.spiders', 'SPIDER_MODULES': ['collectSports.spiders'], 'USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 UBrowser/6.2.4098.3 Safari/537.36'}
- 2019-06-22 10:19:24 [scrapy.extensions.telnet] INFO: Telnet Password: 781e74930d9ccb9f
- 2019-06-22 10:19:24 [scrapy.middleware] INFO: Enabled extensions:
- ['scrapy.extensions.corestats.CoreStats',
- 'scrapy.extensions.telnet.TelnetConsole',
- 'scrapy.extensions.logstats.LogStats']
- 2019-06-22 10:19:24 [scrapy.middleware] INFO: Enabled downloader middlewares:
- ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
- 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
- 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
- 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
- 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
- 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
- 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
- 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
- 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
- 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
- 'scrapy.downloadermiddlewares.stats.DownloaderStats']
- 2019-06-22 10:19:24 [scrapy.middleware] INFO: Enabled spider middlewares:
- ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
- 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
- 'scrapy.spidermiddlewares.referer.RefererMiddleware',
- 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
- 'scrapy.spidermiddlewares.depth.DepthMiddleware']
- 2019-06-22 10:19:24 [scrapy.middleware] INFO: Enabled item pipelines:
- ['collectSports.pipelines.sportslst.SportslstPipeline']
- 2019-06-22 10:19:24 [scrapy.core.engine] INFO: Spider opened
- 2019-06-22 10:19:24 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
- 2019-06-22 10:19:24 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
- 2019-06-22 10:19:25 [scrapy.core.downloader.tls] WARNING: Ignoring error while verifying certificate from host "205.201.4.177" (exception: ValueError('Invalid DNS-ID.'))
- 2019-06-22 10:20:49 [scrapy.utils.log] INFO: Scrapy 1.6.0 started (bot: SportsSpider)
- 2019-06-22 10:20:49 [scrapy.utils.log] INFO: Versions: lxml 4.3.2.0, libxml2 2.9.9, cssselect 1.0.3, parsel 1.5.1, w3lib 1.20.0, Twisted 19.2.1, Python 3.7.3 (default, Mar 27 2019, 17:13:21) [MSC v.1915 64 bit (AMD64)], pyOpenSSL 19.0.0 (OpenSSL 1.1.1b 26 Feb 2019), cryptography 2.6.1, Platform Windows-10-10.0.17763-SP0
- 2019-06-22 10:20:49 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'SportsSpider', 'LOG_FILE': 'SportsSpiderlog20190622.txt', 'LOG_LEVEL': 'INFO', 'NEWSPIDER_MODULE': 'collectSports.spiders', 'SPIDER_MODULES': ['collectSports.spiders'], 'USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 UBrowser/6.2.4098.3 Safari/537.36'}
- 2019-06-22 10:20:49 [scrapy.extensions.telnet] INFO: Telnet Password: dff90327902ecc8a
- 2019-06-22 10:20:49 [scrapy.middleware] INFO: Enabled extensions:
- ['scrapy.extensions.corestats.CoreStats',
- 'scrapy.extensions.telnet.TelnetConsole',
- 'scrapy.extensions.logstats.LogStats']
- 2019-06-22 10:20:49 [scrapy.middleware] INFO: Enabled downloader middlewares:
- ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
- 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
- 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
- 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
- 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
- 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
- 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
- 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
- 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
- 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
- 'scrapy.downloadermiddlewares.stats.DownloaderStats']
- 2019-06-22 10:20:49 [scrapy.middleware] INFO: Enabled spider middlewares:
- ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
- 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
- 'scrapy.spidermiddlewares.referer.RefererMiddleware',
- 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
- 'scrapy.spidermiddlewares.depth.DepthMiddleware']
- 2019-06-22 10:20:49 [scrapy.middleware] INFO: Enabled item pipelines:
- ['collectSports.pipelines.sportslst.SportslstPipeline']
- 2019-06-22 10:20:49 [scrapy.core.engine] INFO: Spider opened
- 2019-06-22 10:20:49 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
- 2019-06-22 10:20:49 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
- 2019-06-22 10:20:50 [scrapy.core.downloader.tls] WARNING: Ignoring error while verifying certificate from host "205.201.4.177" (exception: ValueError('Invalid DNS-ID.'))
- 2019-06-22 10:21:29 [scrapy.utils.log] INFO: Scrapy 1.6.0 started (bot: SportsSpider)
- 2019-06-22 10:21:29 [scrapy.utils.log] INFO: Versions: lxml 4.3.2.0, libxml2 2.9.9, cssselect 1.0.3, parsel 1.5.1, w3lib 1.20.0, Twisted 19.2.1, Python 3.7.3 (default, Mar 27 2019, 17:13:21) [MSC v.1915 64 bit (AMD64)], pyOpenSSL 19.0.0 (OpenSSL 1.1.1b 26 Feb 2019), cryptography 2.6.1, Platform Windows-10-10.0.17763-SP0
- 2019-06-22 10:21:29 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'SportsSpider', 'LOG_FILE': 'SportsSpiderlog20190622.txt', 'LOG_LEVEL': 'INFO', 'NEWSPIDER_MODULE': 'collectSports.spiders', 'SPIDER_MODULES': ['collectSports.spiders'], 'USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 UBrowser/6.2.4098.3 Safari/537.36'}
- 2019-06-22 10:21:29 [scrapy.extensions.telnet] INFO: Telnet Password: 22e1efea74b94993
- 2019-06-22 10:21:29 [scrapy.middleware] INFO: Enabled extensions:
- ['scrapy.extensions.corestats.CoreStats',
- 'scrapy.extensions.telnet.TelnetConsole',
- 'scrapy.extensions.logstats.LogStats']
- 2019-06-22 10:21:29 [scrapy.middleware] INFO: Enabled downloader middlewares:
- ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
- 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
- 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
- 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
- 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
- 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
- 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
- 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
- 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
- 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
- 'scrapy.downloadermiddlewares.stats.DownloaderStats']
- 2019-06-22 10:21:29 [scrapy.middleware] INFO: Enabled spider middlewares:
- ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
- 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
- 'scrapy.spidermiddlewares.referer.RefererMiddleware',
- 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
- 'scrapy.spidermiddlewares.depth.DepthMiddleware']
- 2019-06-22 10:21:29 [scrapy.middleware] INFO: Enabled item pipelines:
- ['collectSports.pipelines.sportslst.SportslstPipeline']
- 2019-06-22 10:21:29 [scrapy.core.engine] INFO: Spider opened
- 2019-06-22 10:21:29 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
- 2019-06-22 10:21:29 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
- 2019-06-22 10:21:30 [scrapy.core.downloader.tls] WARNING: Ignoring error while verifying certificate from host "205.201.4.177" (exception: ValueError('Invalid DNS-ID.'))
- 2019-06-22 10:28:43 [scrapy.utils.log] INFO: Scrapy 1.6.0 started (bot: SportsSpider)
- 2019-06-22 10:28:43 [scrapy.utils.log] INFO: Versions: lxml 4.3.2.0, libxml2 2.9.9, cssselect 1.0.3, parsel 1.5.1, w3lib 1.20.0, Twisted 19.2.1, Python 3.7.3 (default, Mar 27 2019, 17:13:21) [MSC v.1915 64 bit (AMD64)], pyOpenSSL 19.0.0 (OpenSSL 1.1.1b 26 Feb 2019), cryptography 2.6.1, Platform Windows-10-10.0.17763-SP0
- 2019-06-22 10:28:43 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'SportsSpider', 'LOG_FILE': 'SportsSpiderlog20190622.txt', 'LOG_LEVEL': 'INFO', 'NEWSPIDER_MODULE': 'collectSports.spiders', 'SPIDER_MODULES': ['collectSports.spiders'], 'USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 UBrowser/6.2.4098.3 Safari/537.36'}
- 2019-06-22 10:28:43 [scrapy.extensions.telnet] INFO: Telnet Password: 5335f7f9ba6ceccb
- 2019-06-22 10:28:43 [scrapy.middleware] INFO: Enabled extensions:
- ['scrapy.extensions.corestats.CoreStats',
- 'scrapy.extensions.telnet.TelnetConsole',
- 'scrapy.extensions.logstats.LogStats']
- 2019-06-22 10:28:43 [scrapy.middleware] INFO: Enabled downloader middlewares:
- ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
- 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
- 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
- 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
- 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
- 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
- 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
- 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
- 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
- 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
- 'scrapy.downloadermiddlewares.stats.DownloaderStats']
- 2019-06-22 10:28:43 [scrapy.middleware] INFO: Enabled spider middlewares:
- ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
- 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
- 'scrapy.spidermiddlewares.referer.RefererMiddleware',
- 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
- 'scrapy.spidermiddlewares.depth.DepthMiddleware']
- 2019-06-22 10:28:44 [scrapy.middleware] INFO: Enabled item pipelines:
- ['collectSports.pipelines.sportslst.SportslstPipeline']
- 2019-06-22 10:28:44 [scrapy.core.engine] INFO: Spider opened
- 2019-06-22 10:28:44 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
- 2019-06-22 10:28:44 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
- 2019-06-22 10:30:25 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
- 2019-06-22 10:30:27 [scrapy.core.downloader.tls] WARNING: Ignoring error while verifying certificate from host "205.201.4.177" (exception: ValueError('Invalid DNS-ID.'))
- 2019-06-22 10:30:32 [scrapy.utils.log] INFO: Scrapy 1.6.0 started (bot: SportsSpider)
- 2019-06-22 10:30:32 [scrapy.utils.log] INFO: Versions: lxml 4.3.2.0, libxml2 2.9.9, cssselect 1.0.3, parsel 1.5.1, w3lib 1.20.0, Twisted 19.2.1, Python 3.7.3 (default, Mar 27 2019, 17:13:21) [MSC v.1915 64 bit (AMD64)], pyOpenSSL 19.0.0 (OpenSSL 1.1.1b 26 Feb 2019), cryptography 2.6.1, Platform Windows-10-10.0.17763-SP0
- 2019-06-22 10:30:32 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'SportsSpider', 'LOG_FILE': 'SportsSpiderlog20190622.txt', 'LOG_LEVEL': 'INFO', 'NEWSPIDER_MODULE': 'collectSports.spiders', 'SPIDER_MODULES': ['collectSports.spiders'], 'USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 UBrowser/6.2.4098.3 Safari/537.36'}
- 2019-06-22 10:30:32 [scrapy.extensions.telnet] INFO: Telnet Password: 3562166de1a049bb
- 2019-06-22 10:30:32 [scrapy.middleware] INFO: Enabled extensions:
- ['scrapy.extensions.corestats.CoreStats',
- 'scrapy.extensions.telnet.TelnetConsole',
- 'scrapy.extensions.logstats.LogStats']
- 2019-06-22 10:30:32 [scrapy.middleware] INFO: Enabled downloader middlewares:
- ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
- 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
- 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
- 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
- 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
- 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
- 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
- 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
- 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
- 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
- 'scrapy.downloadermiddlewares.stats.DownloaderStats']
- 2019-06-22 10:30:32 [scrapy.middleware] INFO: Enabled spider middlewares:
- ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
- 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
- 'scrapy.spidermiddlewares.referer.RefererMiddleware',
- 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
- 'scrapy.spidermiddlewares.depth.DepthMiddleware']
- 2019-06-22 10:30:32 [scrapy.middleware] INFO: Enabled item pipelines:
- ['collectSports.pipelines.sportslst.SportslstPipeline']
- 2019-06-22 10:30:32 [scrapy.core.engine] INFO: Spider opened
- 2019-06-22 10:30:32 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
- 2019-06-22 10:30:32 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
- 2019-06-22 10:30:35 [scrapy.core.downloader.tls] WARNING: Ignoring error while verifying certificate from host "205.201.4.177" (exception: ValueError('Invalid DNS-ID.'))
- 2019-06-22 10:31:50 [scrapy.utils.log] INFO: Scrapy 1.6.0 started (bot: SportsSpider)
- 2019-06-22 10:31:50 [scrapy.utils.log] INFO: Versions: lxml 4.3.2.0, libxml2 2.9.9, cssselect 1.0.3, parsel 1.5.1, w3lib 1.20.0, Twisted 19.2.1, Python 3.7.3 (default, Mar 27 2019, 17:13:21) [MSC v.1915 64 bit (AMD64)], pyOpenSSL 19.0.0 (OpenSSL 1.1.1b 26 Feb 2019), cryptography 2.6.1, Platform Windows-10-10.0.17763-SP0
- 2019-06-22 10:31:50 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'SportsSpider', 'LOG_FILE': 'SportsSpiderlog20190622.txt', 'LOG_LEVEL': 'INFO', 'NEWSPIDER_MODULE': 'collectSports.spiders', 'SPIDER_MODULES': ['collectSports.spiders'], 'USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 UBrowser/6.2.4098.3 Safari/537.36'}
- 2019-06-22 10:31:50 [scrapy.extensions.telnet] INFO: Telnet Password: a80b9362da564a2d
- 2019-06-22 10:31:50 [scrapy.middleware] INFO: Enabled extensions:
- ['scrapy.extensions.corestats.CoreStats',
- 'scrapy.extensions.telnet.TelnetConsole',
- 'scrapy.extensions.logstats.LogStats']
- 2019-06-22 10:31:50 [scrapy.middleware] INFO: Enabled downloader middlewares:
- ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
- 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
- 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
- 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
- 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
- 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
- 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
- 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
- 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
- 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
- 'scrapy.downloadermiddlewares.stats.DownloaderStats']
- 2019-06-22 10:31:50 [scrapy.middleware] INFO: Enabled spider middlewares:
- ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
- 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
- 'scrapy.spidermiddlewares.referer.RefererMiddleware',
- 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
- 'scrapy.spidermiddlewares.depth.DepthMiddleware']
- 2019-06-22 10:31:51 [scrapy.middleware] INFO: Enabled item pipelines:
- ['collectSports.pipelines.sportslst.SportslstPipeline']
- 2019-06-22 10:31:51 [scrapy.core.engine] INFO: Spider opened
- 2019-06-22 10:31:51 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
- 2019-06-22 10:31:51 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
- 2019-06-22 10:32:22 [scrapy.utils.log] INFO: Scrapy 1.6.0 started (bot: SportsSpider)
- 2019-06-22 10:32:22 [scrapy.utils.log] INFO: Versions: lxml 4.3.2.0, libxml2 2.9.9, cssselect 1.0.3, parsel 1.5.1, w3lib 1.20.0, Twisted 19.2.1, Python 3.7.3 (default, Mar 27 2019, 17:13:21) [MSC v.1915 64 bit (AMD64)], pyOpenSSL 19.0.0 (OpenSSL 1.1.1b 26 Feb 2019), cryptography 2.6.1, Platform Windows-10-10.0.17763-SP0
- 2019-06-22 10:32:22 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'SportsSpider', 'LOG_FILE': 'SportsSpiderlog20190622.txt', 'LOG_LEVEL': 'INFO', 'NEWSPIDER_MODULE': 'collectSports.spiders', 'SPIDER_MODULES': ['collectSports.spiders'], 'USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 UBrowser/6.2.4098.3 Safari/537.36'}
- 2019-06-22 10:32:22 [scrapy.extensions.telnet] INFO: Telnet Password: e0063c1c4b2a77a8
- 2019-06-22 10:32:22 [scrapy.middleware] INFO: Enabled extensions:
- ['scrapy.extensions.corestats.CoreStats',
- 'scrapy.extensions.telnet.TelnetConsole',
- 'scrapy.extensions.logstats.LogStats']
- 2019-06-22 10:32:22 [scrapy.middleware] INFO: Enabled downloader middlewares:
- ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
- 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
- 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
- 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
- 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
- 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
- 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
- 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
- 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
- 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
- 'scrapy.downloadermiddlewares.stats.DownloaderStats']
- 2019-06-22 10:32:22 [scrapy.middleware] INFO: Enabled spider middlewares:
- ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
- 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
- 'scrapy.spidermiddlewares.referer.RefererMiddleware',
- 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
- 'scrapy.spidermiddlewares.depth.DepthMiddleware']
- 2019-06-22 10:32:22 [scrapy.middleware] INFO: Enabled item pipelines:
- ['collectSports.pipelines.sportslst.SportslstPipeline']
- 2019-06-22 10:32:22 [scrapy.core.engine] INFO: Spider opened
- 2019-06-22 10:32:22 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
- 2019-06-22 10:32:22 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
- 2019-06-22 10:34:24 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
- 2019-06-22 10:34:26 [scrapy.core.downloader.tls] WARNING: Ignoring error while verifying certificate from host "205.201.4.177" (exception: ValueError('Invalid DNS-ID.'))
- 2019-06-22 10:35:45 [scrapy.utils.log] INFO: Scrapy 1.6.0 started (bot: SportsSpider)
- 2019-06-22 10:35:45 [scrapy.utils.log] INFO: Versions: lxml 4.3.2.0, libxml2 2.9.9, cssselect 1.0.3, parsel 1.5.1, w3lib 1.20.0, Twisted 19.2.1, Python 3.7.3 (default, Mar 27 2019, 17:13:21) [MSC v.1915 64 bit (AMD64)], pyOpenSSL 19.0.0 (OpenSSL 1.1.1b 26 Feb 2019), cryptography 2.6.1, Platform Windows-10-10.0.17763-SP0
- 2019-06-22 10:35:45 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'SportsSpider', 'LOG_FILE': 'SportsSpiderlog20190622.txt', 'LOG_LEVEL': 'INFO', 'NEWSPIDER_MODULE': 'collectSports.spiders', 'SPIDER_MODULES': ['collectSports.spiders'], 'USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 UBrowser/6.2.4098.3 Safari/537.36'}
- 2019-06-22 10:35:45 [scrapy.extensions.telnet] INFO: Telnet Password: f0012371dcfde2ce
- 2019-06-22 10:35:45 [scrapy.middleware] INFO: Enabled extensions:
- ['scrapy.extensions.corestats.CoreStats',
- 'scrapy.extensions.telnet.TelnetConsole',
- 'scrapy.extensions.logstats.LogStats']
- 2019-06-22 10:35:45 [scrapy.middleware] INFO: Enabled downloader middlewares:
- ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
- 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
- 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
- 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
- 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
- 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
- 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
- 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
- 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
- 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
- 'scrapy.downloadermiddlewares.stats.DownloaderStats']
- 2019-06-22 10:35:45 [scrapy.middleware] INFO: Enabled spider middlewares:
- ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
- 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
- 'scrapy.spidermiddlewares.referer.RefererMiddleware',
- 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
- 'scrapy.spidermiddlewares.depth.DepthMiddleware']
- 2019-06-22 10:35:45 [scrapy.middleware] INFO: Enabled item pipelines:
- ['collectSports.pipelines.sportslst.SportslstPipeline']
- 2019-06-22 10:35:45 [scrapy.core.engine] INFO: Spider opened
- 2019-06-22 10:35:45 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
- 2019-06-22 10:35:45 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
- 2019-06-22 10:35:49 [scrapy.core.downloader.tls] WARNING: Ignoring error while verifying certificate from host "199.26.100.178" (exception: ValueError('Invalid DNS-ID.'))
- 2019-06-22 10:37:07 [scrapy.utils.log] INFO: Scrapy 1.6.0 started (bot: SportsSpider)
- 2019-06-22 10:37:07 [scrapy.utils.log] INFO: Versions: lxml 4.3.2.0, libxml2 2.9.9, cssselect 1.0.3, parsel 1.5.1, w3lib 1.20.0, Twisted 19.2.1, Python 3.7.3 (default, Mar 27 2019, 17:13:21) [MSC v.1915 64 bit (AMD64)], pyOpenSSL 19.0.0 (OpenSSL 1.1.1b 26 Feb 2019), cryptography 2.6.1, Platform Windows-10-10.0.17763-SP0
- 2019-06-22 10:37:07 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'SportsSpider', 'LOG_FILE': 'SportsSpiderlog20190622.txt', 'LOG_LEVEL': 'INFO', 'NEWSPIDER_MODULE': 'collectSports.spiders', 'SPIDER_MODULES': ['collectSports.spiders'], 'USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 UBrowser/6.2.4098.3 Safari/537.36'}
- 2019-06-22 10:37:07 [scrapy.extensions.telnet] INFO: Telnet Password: 0bb9e65abadbab2e
- 2019-06-22 10:37:07 [scrapy.middleware] INFO: Enabled extensions:
- ['scrapy.extensions.corestats.CoreStats',
- 'scrapy.extensions.telnet.TelnetConsole',
- 'scrapy.extensions.logstats.LogStats']
- 2019-06-22 10:37:07 [scrapy.middleware] INFO: Enabled downloader middlewares:
- ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
- 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
- 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
- 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
- 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
- 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
- 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
- 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
- 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
- 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
- 'scrapy.downloadermiddlewares.stats.DownloaderStats']
- 2019-06-22 10:37:07 [scrapy.middleware] INFO: Enabled spider middlewares:
- ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
- 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
- 'scrapy.spidermiddlewares.referer.RefererMiddleware',
- 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
- 'scrapy.spidermiddlewares.depth.DepthMiddleware']
- 2019-06-22 10:37:07 [scrapy.middleware] INFO: Enabled item pipelines:
- ['collectSports.pipelines.sportslst.SportslstPipeline']
- 2019-06-22 10:37:07 [scrapy.core.engine] INFO: Spider opened
- 2019-06-22 10:37:07 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
- 2019-06-22 10:37:07 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
- 2019-06-22 10:48:43 [scrapy.utils.log] INFO: Scrapy 1.6.0 started (bot: SportsSpider)
- 2019-06-22 10:48:43 [scrapy.utils.log] INFO: Versions: lxml 4.3.2.0, libxml2 2.9.9, cssselect 1.0.3, parsel 1.5.1, w3lib 1.20.0, Twisted 19.2.1, Python 3.7.3 (default, Mar 27 2019, 17:13:21) [MSC v.1915 64 bit (AMD64)], pyOpenSSL 19.0.0 (OpenSSL 1.1.1b 26 Feb 2019), cryptography 2.6.1, Platform Windows-10-10.0.17763-SP0
- 2019-06-22 10:48:43 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'SportsSpider', 'LOG_FILE': 'SportsSpiderlog20190622.txt', 'LOG_LEVEL': 'INFO', 'NEWSPIDER_MODULE': 'collectSports.spiders', 'SPIDER_MODULES': ['collectSports.spiders'], 'USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 UBrowser/6.2.4098.3 Safari/537.36'}
- 2019-06-22 10:48:43 [scrapy.extensions.telnet] INFO: Telnet Password: 4317ff83287baaf1
- 2019-06-22 10:48:43 [scrapy.middleware] INFO: Enabled extensions:
- ['scrapy.extensions.corestats.CoreStats',
- 'scrapy.extensions.telnet.TelnetConsole',
- 'scrapy.extensions.logstats.LogStats']
- 2019-06-22 10:48:43 [scrapy.middleware] INFO: Enabled downloader middlewares:
- ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
- 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
- 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
- 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
- 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
- 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
- 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
- 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
- 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
- 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
- 'scrapy.downloadermiddlewares.stats.DownloaderStats']
- 2019-06-22 10:48:43 [scrapy.middleware] INFO: Enabled spider middlewares:
- ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
- 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
- 'scrapy.spidermiddlewares.referer.RefererMiddleware',
- 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
- 'scrapy.spidermiddlewares.depth.DepthMiddleware']
- 2019-06-22 10:48:43 [scrapy.middleware] INFO: Enabled item pipelines:
- ['collectSports.pipelines.sportslst.SportslstPipeline']
- 2019-06-22 10:48:43 [scrapy.core.engine] INFO: Spider opened
- 2019-06-22 10:48:43 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
- 2019-06-22 10:48:43 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
- 2019-06-22 10:49:02 [scrapy.utils.log] INFO: Scrapy 1.6.0 started (bot: SportsSpider)
- 2019-06-22 10:49:02 [scrapy.utils.log] INFO: Versions: lxml 4.3.2.0, libxml2 2.9.9, cssselect 1.0.3, parsel 1.5.1, w3lib 1.20.0, Twisted 19.2.1, Python 3.7.3 (default, Mar 27 2019, 17:13:21) [MSC v.1915 64 bit (AMD64)], pyOpenSSL 19.0.0 (OpenSSL 1.1.1b 26 Feb 2019), cryptography 2.6.1, Platform Windows-10-10.0.17763-SP0
- 2019-06-22 10:49:02 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'SportsSpider', 'LOG_FILE': 'SportsSpiderlog20190622.txt', 'LOG_LEVEL': 'INFO', 'NEWSPIDER_MODULE': 'collectSports.spiders', 'SPIDER_MODULES': ['collectSports.spiders'], 'USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 UBrowser/6.2.4098.3 Safari/537.36'}
- 2019-06-22 10:49:02 [scrapy.extensions.telnet] INFO: Telnet Password: 25bdf773b3876abc
- 2019-06-22 10:49:02 [scrapy.middleware] INFO: Enabled extensions:
- ['scrapy.extensions.corestats.CoreStats',
- 'scrapy.extensions.telnet.TelnetConsole',
- 'scrapy.extensions.logstats.LogStats']
- 2019-06-22 10:49:03 [scrapy.middleware] INFO: Enabled downloader middlewares:
- ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
- 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
- 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
- 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
- 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
- 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
- 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
- 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
- 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
- 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
- 'scrapy.downloadermiddlewares.stats.DownloaderStats']
- 2019-06-22 10:49:03 [scrapy.middleware] INFO: Enabled spider middlewares:
- ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
- 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
- 'scrapy.spidermiddlewares.referer.RefererMiddleware',
- 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
- 'scrapy.spidermiddlewares.depth.DepthMiddleware']
- 2019-06-22 10:49:03 [scrapy.middleware] INFO: Enabled item pipelines:
- ['collectSports.pipelines.sportslst.SportslstPipeline']
- 2019-06-22 10:49:03 [scrapy.core.engine] INFO: Spider opened
- 2019-06-22 10:49:03 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
- 2019-06-22 10:49:03 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
- 2019-06-22 10:49:09 [scrapy.core.downloader.tls] WARNING: Ignoring error while verifying certificate from host "199.26.100.178" (exception: ValueError('Invalid DNS-ID.'))
- 2019-06-22 10:49:45 [scrapy.utils.log] INFO: Scrapy 1.6.0 started (bot: SportsSpider)
- 2019-06-22 10:49:45 [scrapy.utils.log] INFO: Versions: lxml 4.3.2.0, libxml2 2.9.9, cssselect 1.0.3, parsel 1.5.1, w3lib 1.20.0, Twisted 19.2.1, Python 3.7.3 (default, Mar 27 2019, 17:13:21) [MSC v.1915 64 bit (AMD64)], pyOpenSSL 19.0.0 (OpenSSL 1.1.1b 26 Feb 2019), cryptography 2.6.1, Platform Windows-10-10.0.17763-SP0
- 2019-06-22 10:49:45 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'SportsSpider', 'LOG_FILE': 'SportsSpiderlog20190622.txt', 'LOG_LEVEL': 'INFO', 'NEWSPIDER_MODULE': 'collectSports.spiders', 'SPIDER_MODULES': ['collectSports.spiders'], 'USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 UBrowser/6.2.4098.3 Safari/537.36'}
- 2019-06-22 10:49:45 [scrapy.extensions.telnet] INFO: Telnet Password: 6c328f0307217b75
- 2019-06-22 10:49:45 [scrapy.middleware] INFO: Enabled extensions:
- ['scrapy.extensions.corestats.CoreStats',
- 'scrapy.extensions.telnet.TelnetConsole',
- 'scrapy.extensions.logstats.LogStats']
- 2019-06-22 10:49:45 [scrapy.middleware] INFO: Enabled downloader middlewares:
- ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
- 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
- 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
- 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
- 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
- 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
- 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
- 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
- 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
- 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
- 'scrapy.downloadermiddlewares.stats.DownloaderStats']
- 2019-06-22 10:49:45 [scrapy.middleware] INFO: Enabled spider middlewares:
- ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
- 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
- 'scrapy.spidermiddlewares.referer.RefererMiddleware',
- 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
- 'scrapy.spidermiddlewares.depth.DepthMiddleware']
- 2019-06-22 10:49:45 [scrapy.middleware] INFO: Enabled item pipelines:
- ['collectSports.pipelines.sportslst.SportslstPipeline']
- 2019-06-22 10:49:45 [scrapy.core.engine] INFO: Spider opened
- 2019-06-22 10:49:45 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
- 2019-06-22 10:49:45 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
- 2019-06-22 10:49:57 [scrapy.utils.log] INFO: Scrapy 1.6.0 started (bot: SportsSpider)
- 2019-06-22 10:49:57 [scrapy.utils.log] INFO: Versions: lxml 4.3.2.0, libxml2 2.9.9, cssselect 1.0.3, parsel 1.5.1, w3lib 1.20.0, Twisted 19.2.1, Python 3.7.3 (default, Mar 27 2019, 17:13:21) [MSC v.1915 64 bit (AMD64)], pyOpenSSL 19.0.0 (OpenSSL 1.1.1b 26 Feb 2019), cryptography 2.6.1, Platform Windows-10-10.0.17763-SP0
- 2019-06-22 10:49:57 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'SportsSpider', 'LOG_FILE': 'SportsSpiderlog20190622.txt', 'LOG_LEVEL': 'INFO', 'NEWSPIDER_MODULE': 'collectSports.spiders', 'SPIDER_MODULES': ['collectSports.spiders'], 'USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 UBrowser/6.2.4098.3 Safari/537.36'}
- 2019-06-22 10:49:57 [scrapy.extensions.telnet] INFO: Telnet Password: 4fb03eca233bcca3
- 2019-06-22 10:49:57 [scrapy.middleware] INFO: Enabled extensions:
- ['scrapy.extensions.corestats.CoreStats',
- 'scrapy.extensions.telnet.TelnetConsole',
- 'scrapy.extensions.logstats.LogStats']
- 2019-06-22 10:49:58 [scrapy.middleware] INFO: Enabled downloader middlewares:
- ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
- 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
- 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
- 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
- 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
- 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
- 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
- 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
- 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
- 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
- 'scrapy.downloadermiddlewares.stats.DownloaderStats']
- 2019-06-22 10:49:58 [scrapy.middleware] INFO: Enabled spider middlewares:
- ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
- 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
- 'scrapy.spidermiddlewares.referer.RefererMiddleware',
- 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
- 'scrapy.spidermiddlewares.depth.DepthMiddleware']
- 2019-06-22 10:49:58 [scrapy.middleware] INFO: Enabled item pipelines:
- ['collectSports.pipelines.sportslst.SportslstPipeline']
- 2019-06-22 10:49:58 [scrapy.core.engine] INFO: Spider opened
- 2019-06-22 10:49:58 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
- 2019-06-22 10:49:58 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
- 2019-06-22 10:50:03 [scrapy.core.downloader.tls] WARNING: Ignoring error while verifying certificate from host "205.201.4.177" (exception: ValueError('Invalid DNS-ID.'))
- 2019-06-22 10:50:56 [scrapy.utils.log] INFO: Scrapy 1.6.0 started (bot: SportsSpider)
- 2019-06-22 10:50:56 [scrapy.utils.log] INFO: Versions: lxml 4.3.2.0, libxml2 2.9.9, cssselect 1.0.3, parsel 1.5.1, w3lib 1.20.0, Twisted 19.2.1, Python 3.7.3 (default, Mar 27 2019, 17:13:21) [MSC v.1915 64 bit (AMD64)], pyOpenSSL 19.0.0 (OpenSSL 1.1.1b 26 Feb 2019), cryptography 2.6.1, Platform Windows-10-10.0.17763-SP0
- 2019-06-22 10:50:56 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'SportsSpider', 'LOG_FILE': 'SportsSpiderlog20190622.txt', 'LOG_LEVEL': 'INFO', 'NEWSPIDER_MODULE': 'collectSports.spiders', 'SPIDER_MODULES': ['collectSports.spiders'], 'USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 UBrowser/6.2.4098.3 Safari/537.36'}
- 2019-06-22 10:50:56 [scrapy.extensions.telnet] INFO: Telnet Password: 66cc7c4c24cc3cae
- 2019-06-22 10:50:56 [scrapy.middleware] INFO: Enabled extensions:
- ['scrapy.extensions.corestats.CoreStats',
- 'scrapy.extensions.telnet.TelnetConsole',
- 'scrapy.extensions.logstats.LogStats']
- 2019-06-22 10:50:56 [scrapy.middleware] INFO: Enabled downloader middlewares:
- ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
- 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
- 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
- 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
- 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
- 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
- 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
- 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
- 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
- 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
- 'scrapy.downloadermiddlewares.stats.DownloaderStats']
- 2019-06-22 10:50:56 [scrapy.middleware] INFO: Enabled spider middlewares:
- ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
- 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
- 'scrapy.spidermiddlewares.referer.RefererMiddleware',
- 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
- 'scrapy.spidermiddlewares.depth.DepthMiddleware']
- 2019-06-22 10:50:56 [scrapy.middleware] INFO: Enabled item pipelines:
- ['collectSports.pipelines.sportslst.SportslstPipeline']
- 2019-06-22 10:50:56 [scrapy.core.engine] INFO: Spider opened
- 2019-06-22 10:50:56 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
- 2019-06-22 10:50:56 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
- 2019-06-22 10:50:59 [scrapy.core.downloader.tls] WARNING: Ignoring error while verifying certificate from host "205.201.4.177" (exception: ValueError('Invalid DNS-ID.'))
- 2019-06-22 11:04:38 [scrapy.utils.log] INFO: Scrapy 1.6.0 started (bot: SportsSpider)
- 2019-06-22 11:04:38 [scrapy.utils.log] INFO: Versions: lxml 4.3.2.0, libxml2 2.9.9, cssselect 1.0.3, parsel 1.5.1, w3lib 1.20.0, Twisted 19.2.1, Python 3.7.3 (default, Mar 27 2019, 17:13:21) [MSC v.1915 64 bit (AMD64)], pyOpenSSL 19.0.0 (OpenSSL 1.1.1b 26 Feb 2019), cryptography 2.6.1, Platform Windows-10-10.0.17763-SP0
- 2019-06-22 11:04:38 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'SportsSpider', 'LOG_FILE': 'SportsSpiderlog20190622.txt', 'LOG_LEVEL': 'INFO', 'NEWSPIDER_MODULE': 'collectSports.spiders', 'SPIDER_MODULES': ['collectSports.spiders'], 'USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 UBrowser/6.2.4098.3 Safari/537.36'}
- 2019-06-22 11:04:38 [scrapy.extensions.telnet] INFO: Telnet Password: b810244ec17ecefd
- 2019-06-22 11:04:39 [scrapy.middleware] INFO: Enabled extensions:
- ['scrapy.extensions.corestats.CoreStats',
- 'scrapy.extensions.telnet.TelnetConsole',
- 'scrapy.extensions.logstats.LogStats']
- 2019-06-22 11:04:39 [scrapy.middleware] INFO: Enabled downloader middlewares:
- ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
- 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
- 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
- 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
- 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
- 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
- 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
- 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
- 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
- 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
- 'scrapy.downloadermiddlewares.stats.DownloaderStats']
- 2019-06-22 11:04:39 [scrapy.middleware] INFO: Enabled spider middlewares:
- ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
- 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
- 'scrapy.spidermiddlewares.referer.RefererMiddleware',
- 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
- 'scrapy.spidermiddlewares.depth.DepthMiddleware']
- 2019-06-22 11:04:39 [scrapy.middleware] INFO: Enabled item pipelines:
- ['collectSports.pipelines.sportslst.SportslstPipeline']
- 2019-06-22 11:04:39 [scrapy.core.engine] INFO: Spider opened
- 2019-06-22 11:04:39 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
- 2019-06-22 11:04:39 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
- 2019-06-22 11:05:41 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
- 2019-06-22 11:05:41 [scrapy.core.downloader.tls] WARNING: Ignoring error while verifying certificate from host "205.201.4.177" (exception: ValueError('Invalid DNS-ID.'))
- 2019-06-22 11:09:12 [scrapy.utils.log] INFO: Scrapy 1.6.0 started (bot: SportsSpider)
- 2019-06-22 11:09:12 [scrapy.utils.log] INFO: Versions: lxml 4.3.2.0, libxml2 2.9.9, cssselect 1.0.3, parsel 1.5.1, w3lib 1.20.0, Twisted 19.2.1, Python 3.7.3 (default, Mar 27 2019, 17:13:21) [MSC v.1915 64 bit (AMD64)], pyOpenSSL 19.0.0 (OpenSSL 1.1.1b 26 Feb 2019), cryptography 2.6.1, Platform Windows-10-10.0.17763-SP0
- 2019-06-22 11:09:12 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'SportsSpider', 'LOG_FILE': 'SportsSpiderlog20190622.txt', 'LOG_LEVEL': 'INFO', 'NEWSPIDER_MODULE': 'collectSports.spiders', 'SPIDER_MODULES': ['collectSports.spiders'], 'USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 UBrowser/6.2.4098.3 Safari/537.36'}
- 2019-06-22 11:09:12 [scrapy.extensions.telnet] INFO: Telnet Password: 7def27b7bb4424ee
- 2019-06-22 11:09:12 [scrapy.middleware] INFO: Enabled extensions:
- ['scrapy.extensions.corestats.CoreStats',
- 'scrapy.extensions.telnet.TelnetConsole',
- 'scrapy.extensions.logstats.LogStats']
- 2019-06-22 11:09:12 [scrapy.middleware] INFO: Enabled downloader middlewares:
- ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
- 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
- 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
- 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
- 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
- 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
- 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
- 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
- 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
- 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
- 'scrapy.downloadermiddlewares.stats.DownloaderStats']
- 2019-06-22 11:09:12 [scrapy.middleware] INFO: Enabled spider middlewares:
- ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
- 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
- 'scrapy.spidermiddlewares.referer.RefererMiddleware',
- 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
- 'scrapy.spidermiddlewares.depth.DepthMiddleware']
- 2019-06-22 11:09:12 [scrapy.middleware] INFO: Enabled item pipelines:
- ['collectSports.pipelines.sportslst.SportslstPipeline']
- 2019-06-22 11:09:12 [scrapy.core.engine] INFO: Spider opened
- 2019-06-22 11:09:12 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
- 2019-06-22 11:09:12 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
- 2019-06-22 11:09:16 [scrapy.core.downloader.tls] WARNING: Ignoring error while verifying certificate from host "205.201.4.177" (exception: ValueError('Invalid DNS-ID.'))
- 2019-06-22 11:12:01 [scrapy.utils.log] INFO: Scrapy 1.6.0 started (bot: SportsSpider)
- 2019-06-22 11:12:01 [scrapy.utils.log] INFO: Versions: lxml 4.3.2.0, libxml2 2.9.9, cssselect 1.0.3, parsel 1.5.1, w3lib 1.20.0, Twisted 19.2.1, Python 3.7.3 (default, Mar 27 2019, 17:13:21) [MSC v.1915 64 bit (AMD64)], pyOpenSSL 19.0.0 (OpenSSL 1.1.1b 26 Feb 2019), cryptography 2.6.1, Platform Windows-10-10.0.17763-SP0
- 2019-06-22 11:12:01 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'SportsSpider', 'LOG_FILE': 'SportsSpiderlog20190622.txt', 'LOG_LEVEL': 'INFO', 'NEWSPIDER_MODULE': 'collectSports.spiders', 'SPIDER_MODULES': ['collectSports.spiders'], 'USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 UBrowser/6.2.4098.3 Safari/537.36'}
- 2019-06-22 11:12:01 [scrapy.extensions.telnet] INFO: Telnet Password: 1985e25100bae467
- 2019-06-22 11:12:01 [scrapy.middleware] INFO: Enabled extensions:
- ['scrapy.extensions.corestats.CoreStats',
- 'scrapy.extensions.telnet.TelnetConsole',
- 'scrapy.extensions.logstats.LogStats']
- 2019-06-22 11:12:01 [scrapy.middleware] INFO: Enabled downloader middlewares:
- ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
- 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
- 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
- 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
- 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
- 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
- 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
- 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
- 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
- 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
- 'scrapy.downloadermiddlewares.stats.DownloaderStats']
- 2019-06-22 11:12:01 [scrapy.middleware] INFO: Enabled spider middlewares:
- ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
- 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
- 'scrapy.spidermiddlewares.referer.RefererMiddleware',
- 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
- 'scrapy.spidermiddlewares.depth.DepthMiddleware']
- 2019-06-22 11:12:01 [scrapy.middleware] INFO: Enabled item pipelines:
- ['collectSports.pipelines.sportslst.SportslstPipeline']
- 2019-06-22 11:12:01 [scrapy.core.engine] INFO: Spider opened
- 2019-06-22 11:12:01 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
- 2019-06-22 11:12:01 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
- 2019-06-22 11:12:11 [scrapy.core.downloader.tls] WARNING: Ignoring error while verifying certificate from host "199.26.100.178" (exception: ValueError('Invalid DNS-ID.'))
- 2019-06-22 11:31:13 [scrapy.utils.log] INFO: Scrapy 1.6.0 started (bot: SportsSpider)
- 2019-06-22 11:31:13 [scrapy.utils.log] INFO: Versions: lxml 4.3.2.0, libxml2 2.9.9, cssselect 1.0.3, parsel 1.5.1, w3lib 1.20.0, Twisted 19.2.1, Python 3.7.3 (default, Mar 27 2019, 17:13:21) [MSC v.1915 64 bit (AMD64)], pyOpenSSL 19.0.0 (OpenSSL 1.1.1b 26 Feb 2019), cryptography 2.6.1, Platform Windows-10-10.0.17763-SP0
- 2019-06-22 11:31:13 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'SportsSpider', 'LOG_FILE': 'SportsSpiderlog20190622.txt', 'LOG_LEVEL': 'INFO', 'NEWSPIDER_MODULE': 'collectSports.spiders', 'SPIDER_MODULES': ['collectSports.spiders'], 'USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 UBrowser/6.2.4098.3 Safari/537.36'}
- 2019-06-22 11:31:13 [scrapy.extensions.telnet] INFO: Telnet Password: 62900213ae5d40a0
- 2019-06-22 11:31:13 [scrapy.middleware] INFO: Enabled extensions:
- ['scrapy.extensions.corestats.CoreStats',
- 'scrapy.extensions.telnet.TelnetConsole',
- 'scrapy.extensions.logstats.LogStats']
- 2019-06-22 11:31:14 [scrapy.middleware] INFO: Enabled downloader middlewares:
- ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
- 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
- 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
- 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
- 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
- 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
- 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
- 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
- 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
- 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
- 'scrapy.downloadermiddlewares.stats.DownloaderStats']
- 2019-06-22 11:31:14 [scrapy.middleware] INFO: Enabled spider middlewares:
- ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
- 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
- 'scrapy.spidermiddlewares.referer.RefererMiddleware',
- 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
- 'scrapy.spidermiddlewares.depth.DepthMiddleware']
- 2019-06-22 11:31:14 [scrapy.middleware] INFO: Enabled item pipelines:
- ['collectSports.pipelines.sportslst.SportslstPipeline']
- 2019-06-22 11:31:14 [scrapy.core.engine] INFO: Spider opened
- 2019-06-22 11:31:14 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
- 2019-06-22 11:31:14 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
- 2019-06-22 11:31:18 [scrapy.core.downloader.tls] WARNING: Ignoring error while verifying certificate from host "205.201.4.177" (exception: ValueError('Invalid DNS-ID.'))
- 2019-06-22 11:31:46 [scrapy.core.engine] INFO: Closing spider (finished)
- 2019-06-22 11:31:46 [scrapy.core.engine] ERROR: Scraper close failure
- Traceback (most recent call last):
- File "C:\ProgramData\Anaconda3\lib\site-packages\twisted\internet\defer.py", line 654, in _runCallbacks
- current.result = callback(current.result, *args, **kw)
- File "D:\workspace\sports_collect\collectSports\pipelines\sportslst.py", line 170, in close_spider
- self.client.close()
- AttributeError: 'SportslstPipeline' object has no attribute 'client'
- 2019-06-22 11:31:46 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
- {'downloader/request_bytes': 455,
- 'downloader/request_count': 1,
- 'downloader/request_method_count/GET': 1,
- 'downloader/response_bytes': 4730,
- 'downloader/response_count': 1,
- 'downloader/response_status_count/200': 1,
- 'finish_reason': 'finished',
- 'finish_time': datetime.datetime(2019, 6, 22, 3, 31, 46, 465865),
- 'item_scraped_count': 1,
- 'log_count/ERROR': 1,
- 'log_count/INFO': 9,
- 'log_count/WARNING': 1,
- 'response_received_count': 1,
- 'scheduler/dequeued': 1,
- 'scheduler/dequeued/memory': 1,
- 'scheduler/enqueued': 1,
- 'scheduler/enqueued/memory': 1,
- 'start_time': datetime.datetime(2019, 6, 22, 3, 31, 14, 150055)}
- 2019-06-22 11:31:46 [scrapy.core.engine] INFO: Spider closed (finished)
- 2019-06-22 11:32:05 [scrapy.utils.log] INFO: Scrapy 1.6.0 started (bot: SportsSpider)
- 2019-06-22 11:32:05 [scrapy.utils.log] INFO: Versions: lxml 4.3.2.0, libxml2 2.9.9, cssselect 1.0.3, parsel 1.5.1, w3lib 1.20.0, Twisted 19.2.1, Python 3.7.3 (default, Mar 27 2019, 17:13:21) [MSC v.1915 64 bit (AMD64)], pyOpenSSL 19.0.0 (OpenSSL 1.1.1b 26 Feb 2019), cryptography 2.6.1, Platform Windows-10-10.0.17763-SP0
- 2019-06-22 11:32:05 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'SportsSpider', 'LOG_FILE': 'SportsSpiderlog20190622.txt', 'LOG_LEVEL': 'INFO', 'NEWSPIDER_MODULE': 'collectSports.spiders', 'SPIDER_MODULES': ['collectSports.spiders'], 'USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 UBrowser/6.2.4098.3 Safari/537.36'}
- 2019-06-22 11:32:05 [scrapy.extensions.telnet] INFO: Telnet Password: 70a29d0d9071d1a7
- 2019-06-22 11:32:05 [scrapy.middleware] INFO: Enabled extensions:
- ['scrapy.extensions.corestats.CoreStats',
- 'scrapy.extensions.telnet.TelnetConsole',
- 'scrapy.extensions.logstats.LogStats']
- 2019-06-22 11:32:06 [scrapy.middleware] INFO: Enabled downloader middlewares:
- ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
- 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
- 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
- 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
- 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
- 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
- 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
- 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
- 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
- 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
- 'scrapy.downloadermiddlewares.stats.DownloaderStats']
- 2019-06-22 11:32:06 [scrapy.middleware] INFO: Enabled spider middlewares:
- ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
- 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
- 'scrapy.spidermiddlewares.referer.RefererMiddleware',
- 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
- 'scrapy.spidermiddlewares.depth.DepthMiddleware']
- 2019-06-22 11:32:06 [scrapy.middleware] INFO: Enabled item pipelines:
- ['collectSports.pipelines.sportslst.SportslstPipeline']
- 2019-06-22 11:32:06 [scrapy.core.engine] INFO: Spider opened
- 2019-06-22 11:32:06 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
- 2019-06-22 11:32:06 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
- 2019-06-22 11:32:06 [scrapy.core.downloader.tls] WARNING: Ignoring error while verifying certificate from host "205.201.4.177" (exception: ValueError('Invalid DNS-ID.'))
- 2019-06-22 11:32:11 [scrapy.core.engine] INFO: Closing spider (finished)
- 2019-06-22 11:32:11 [scrapy.core.engine] ERROR: Scraper close failure
- Traceback (most recent call last):
- File "C:\ProgramData\Anaconda3\lib\site-packages\twisted\internet\defer.py", line 654, in _runCallbacks
- current.result = callback(current.result, *args, **kw)
- File "D:\workspace\sports_collect\collectSports\pipelines\sportslst.py", line 170, in close_spider
- self.client.close()
- AttributeError: 'SportslstPipeline' object has no attribute 'client'
- 2019-06-22 11:32:11 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
- {'downloader/request_bytes': 455,
- 'downloader/request_count': 1,
- 'downloader/request_method_count/GET': 1,
- 'downloader/response_bytes': 4731,
- 'downloader/response_count': 1,
- 'downloader/response_status_count/200': 1,
- 'finish_reason': 'finished',
- 'finish_time': datetime.datetime(2019, 6, 22, 3, 32, 11, 335988),
- 'item_scraped_count': 1,
- 'log_count/ERROR': 1,
- 'log_count/INFO': 9,
- 'log_count/WARNING': 1,
- 'response_received_count': 1,
- 'scheduler/dequeued': 1,
- 'scheduler/dequeued/memory': 1,
- 'scheduler/enqueued': 1,
- 'scheduler/enqueued/memory': 1,
- 'start_time': datetime.datetime(2019, 6, 22, 3, 32, 6, 130610)}
- 2019-06-22 11:32:11 [scrapy.core.engine] INFO: Spider closed (finished)
- 2019-06-22 11:32:27 [scrapy.utils.log] INFO: Scrapy 1.6.0 started (bot: SportsSpider)
- 2019-06-22 11:32:27 [scrapy.utils.log] INFO: Versions: lxml 4.3.2.0, libxml2 2.9.9, cssselect 1.0.3, parsel 1.5.1, w3lib 1.20.0, Twisted 19.2.1, Python 3.7.3 (default, Mar 27 2019, 17:13:21) [MSC v.1915 64 bit (AMD64)], pyOpenSSL 19.0.0 (OpenSSL 1.1.1b 26 Feb 2019), cryptography 2.6.1, Platform Windows-10-10.0.17763-SP0
- 2019-06-22 11:32:27 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'SportsSpider', 'LOG_FILE': 'SportsSpiderlog20190622.txt', 'LOG_LEVEL': 'INFO', 'NEWSPIDER_MODULE': 'collectSports.spiders', 'SPIDER_MODULES': ['collectSports.spiders'], 'USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 UBrowser/6.2.4098.3 Safari/537.36'}
- 2019-06-22 11:32:27 [scrapy.extensions.telnet] INFO: Telnet Password: 64b2924bd6b7e99d
- 2019-06-22 11:32:27 [scrapy.middleware] INFO: Enabled extensions:
- ['scrapy.extensions.corestats.CoreStats',
- 'scrapy.extensions.telnet.TelnetConsole',
- 'scrapy.extensions.logstats.LogStats']
- 2019-06-22 11:32:27 [scrapy.middleware] INFO: Enabled downloader middlewares:
- ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
- 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
- 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
- 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
- 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
- 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
- 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
- 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
- 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
- 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
- 'scrapy.downloadermiddlewares.stats.DownloaderStats']
- 2019-06-22 11:32:27 [scrapy.middleware] INFO: Enabled spider middlewares:
- ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
- 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
- 'scrapy.spidermiddlewares.referer.RefererMiddleware',
- 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
- 'scrapy.spidermiddlewares.depth.DepthMiddleware']
- 2019-06-22 11:32:27 [scrapy.middleware] INFO: Enabled item pipelines:
- ['collectSports.pipelines.sportslst.SportslstPipeline']
- 2019-06-22 11:32:27 [scrapy.core.engine] INFO: Spider opened
- 2019-06-22 11:32:27 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
- 2019-06-22 11:32:27 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
- 2019-06-22 11:32:28 [scrapy.core.downloader.tls] WARNING: Ignoring error while verifying certificate from host "205.201.4.177" (exception: ValueError('Invalid DNS-ID.'))
- 2019-06-22 11:34:48 [scrapy.extensions.logstats] INFO: Crawled 1 pages (at 1 pages/min), scraped 1 items (at 1 items/min)
- 2019-06-22 11:34:48 [scrapy.core.engine] INFO: Closing spider (finished)
- 2019-06-22 11:34:48 [scrapy.core.engine] ERROR: Scraper close failure
- Traceback (most recent call last):
- File "C:\ProgramData\Anaconda3\lib\site-packages\twisted\internet\defer.py", line 654, in _runCallbacks
- current.result = callback(current.result, *args, **kw)
- File "D:\workspace\sports_collect\collectSports\pipelines\sportslst.py", line 170, in close_spider
- self.client.close()
- AttributeError: 'SportslstPipeline' object has no attribute 'client'
- 2019-06-22 11:34:48 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
- {'downloader/request_bytes': 455,
- 'downloader/request_count': 1,
- 'downloader/request_method_count/GET': 1,
- 'downloader/response_bytes': 4729,
- 'downloader/response_count': 1,
- 'downloader/response_status_count/200': 1,
- 'finish_reason': 'finished',
- 'finish_time': datetime.datetime(2019, 6, 22, 3, 34, 48, 972961),
- 'item_scraped_count': 1,
- 'log_count/ERROR': 1,
- 'log_count/INFO': 10,
- 'log_count/WARNING': 1,
- 'response_received_count': 1,
- 'scheduler/dequeued': 1,
- 'scheduler/dequeued/memory': 1,
- 'scheduler/enqueued': 1,
- 'scheduler/enqueued/memory': 1,
- 'start_time': datetime.datetime(2019, 6, 22, 3, 32, 27, 913085)}
- 2019-06-22 11:34:48 [scrapy.core.engine] INFO: Spider closed (finished)
- 2019-06-22 11:35:10 [scrapy.utils.log] INFO: Scrapy 1.6.0 started (bot: SportsSpider)
- 2019-06-22 11:35:10 [scrapy.utils.log] INFO: Versions: lxml 4.3.2.0, libxml2 2.9.9, cssselect 1.0.3, parsel 1.5.1, w3lib 1.20.0, Twisted 19.2.1, Python 3.7.3 (default, Mar 27 2019, 17:13:21) [MSC v.1915 64 bit (AMD64)], pyOpenSSL 19.0.0 (OpenSSL 1.1.1b 26 Feb 2019), cryptography 2.6.1, Platform Windows-10-10.0.17763-SP0
- 2019-06-22 11:35:10 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'SportsSpider', 'LOG_FILE': 'SportsSpiderlog20190622.txt', 'LOG_LEVEL': 'INFO', 'NEWSPIDER_MODULE': 'collectSports.spiders', 'SPIDER_MODULES': ['collectSports.spiders'], 'USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 UBrowser/6.2.4098.3 Safari/537.36'}
- 2019-06-22 11:35:10 [scrapy.extensions.telnet] INFO: Telnet Password: c93ed231f5d9ff4c
- 2019-06-22 11:35:10 [scrapy.middleware] INFO: Enabled extensions:
- ['scrapy.extensions.corestats.CoreStats',
- 'scrapy.extensions.telnet.TelnetConsole',
- 'scrapy.extensions.logstats.LogStats']
- 2019-06-22 11:35:10 [scrapy.middleware] INFO: Enabled downloader middlewares:
- ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
- 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
- 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
- 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
- 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
- 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
- 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
- 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
- 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
- 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
- 'scrapy.downloadermiddlewares.stats.DownloaderStats']
- 2019-06-22 11:35:10 [scrapy.middleware] INFO: Enabled spider middlewares:
- ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
- 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
- 'scrapy.spidermiddlewares.referer.RefererMiddleware',
- 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
- 'scrapy.spidermiddlewares.depth.DepthMiddleware']
- 2019-06-22 11:35:10 [scrapy.middleware] INFO: Enabled item pipelines:
- ['collectSports.pipelines.sportslst.SportslstPipeline']
- 2019-06-22 11:35:10 [scrapy.core.engine] INFO: Spider opened
- 2019-06-22 11:35:10 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
- 2019-06-22 11:35:10 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
- 2019-06-22 11:35:10 [scrapy.core.downloader.tls] WARNING: Ignoring error while verifying certificate from host "205.201.4.177" (exception: ValueError('Invalid DNS-ID.'))
- 2019-06-22 11:36:05 [scrapy.utils.log] INFO: Scrapy 1.6.0 started (bot: SportsSpider)
- 2019-06-22 11:36:05 [scrapy.utils.log] INFO: Versions: lxml 4.3.2.0, libxml2 2.9.9, cssselect 1.0.3, parsel 1.5.1, w3lib 1.20.0, Twisted 19.2.1, Python 3.7.3 (default, Mar 27 2019, 17:13:21) [MSC v.1915 64 bit (AMD64)], pyOpenSSL 19.0.0 (OpenSSL 1.1.1b 26 Feb 2019), cryptography 2.6.1, Platform Windows-10-10.0.17763-SP0
- 2019-06-22 11:36:05 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'SportsSpider', 'LOG_FILE': 'SportsSpiderlog20190622.txt', 'LOG_LEVEL': 'INFO', 'NEWSPIDER_MODULE': 'collectSports.spiders', 'SPIDER_MODULES': ['collectSports.spiders'], 'USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 UBrowser/6.2.4098.3 Safari/537.36'}
- 2019-06-22 11:36:05 [scrapy.extensions.telnet] INFO: Telnet Password: c52ded6dec493710
- 2019-06-22 11:36:05 [scrapy.middleware] INFO: Enabled extensions:
- ['scrapy.extensions.corestats.CoreStats',
- 'scrapy.extensions.telnet.TelnetConsole',
- 'scrapy.extensions.logstats.LogStats']
- 2019-06-22 11:36:05 [scrapy.middleware] INFO: Enabled downloader middlewares:
- ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
- 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
- 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
- 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
- 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
- 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
- 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
- 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
- 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
- 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
- 'scrapy.downloadermiddlewares.stats.DownloaderStats']
- 2019-06-22 11:36:05 [scrapy.middleware] INFO: Enabled spider middlewares:
- ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
- 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
- 'scrapy.spidermiddlewares.referer.RefererMiddleware',
- 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
- 'scrapy.spidermiddlewares.depth.DepthMiddleware']
- 2019-06-22 11:36:05 [scrapy.middleware] INFO: Enabled item pipelines:
- ['collectSports.pipelines.sportslst.SportslstPipeline']
- 2019-06-22 11:36:05 [scrapy.core.engine] INFO: Spider opened
- 2019-06-22 11:36:05 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
- 2019-06-22 11:36:05 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
- 2019-06-22 11:36:05 [scrapy.core.downloader.tls] WARNING: Ignoring error while verifying certificate from host "205.201.4.177" (exception: ValueError('Invalid DNS-ID.'))
- 2019-06-22 11:36:10 [scrapy.core.engine] INFO: Closing spider (finished)
- 2019-06-22 11:36:10 [scrapy.core.engine] ERROR: Scraper close failure
- Traceback (most recent call last):
- File "C:\ProgramData\Anaconda3\lib\site-packages\twisted\internet\defer.py", line 654, in _runCallbacks
- current.result = callback(current.result, *args, **kw)
- File "D:\workspace\sports_collect\collectSports\pipelines\sportslst.py", line 170, in close_spider
- self.client.close()
- AttributeError: 'SportslstPipeline' object has no attribute 'client'
- 2019-06-22 11:36:10 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
- {'downloader/request_bytes': 455,
- 'downloader/request_count': 1,
- 'downloader/request_method_count/GET': 1,
- 'downloader/response_bytes': 4728,
- 'downloader/response_count': 1,
- 'downloader/response_status_count/200': 1,
- 'finish_reason': 'finished',
- 'finish_time': datetime.datetime(2019, 6, 22, 3, 36, 10, 724928),
- 'item_scraped_count': 1,
- 'log_count/ERROR': 1,
- 'log_count/INFO': 9,
- 'log_count/WARNING': 1,
- 'response_received_count': 1,
- 'scheduler/dequeued': 1,
- 'scheduler/dequeued/memory': 1,
- 'scheduler/enqueued': 1,
- 'scheduler/enqueued/memory': 1,
- 'start_time': datetime.datetime(2019, 6, 22, 3, 36, 5, 426056)}
- 2019-06-22 11:36:10 [scrapy.core.engine] INFO: Spider closed (finished)
- 2019-06-22 11:36:27 [scrapy.utils.log] INFO: Scrapy 1.6.0 started (bot: SportsSpider)
- 2019-06-22 11:36:27 [scrapy.utils.log] INFO: Versions: lxml 4.3.2.0, libxml2 2.9.9, cssselect 1.0.3, parsel 1.5.1, w3lib 1.20.0, Twisted 19.2.1, Python 3.7.3 (default, Mar 27 2019, 17:13:21) [MSC v.1915 64 bit (AMD64)], pyOpenSSL 19.0.0 (OpenSSL 1.1.1b 26 Feb 2019), cryptography 2.6.1, Platform Windows-10-10.0.17763-SP0
- 2019-06-22 11:36:27 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'SportsSpider', 'LOG_FILE': 'SportsSpiderlog20190622.txt', 'LOG_LEVEL': 'INFO', 'NEWSPIDER_MODULE': 'collectSports.spiders', 'SPIDER_MODULES': ['collectSports.spiders'], 'USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 UBrowser/6.2.4098.3 Safari/537.36'}
- 2019-06-22 11:36:27 [scrapy.extensions.telnet] INFO: Telnet Password: 35a021e2bc32beed
- 2019-06-22 11:36:27 [scrapy.middleware] INFO: Enabled extensions:
- ['scrapy.extensions.corestats.CoreStats',
- 'scrapy.extensions.telnet.TelnetConsole',
- 'scrapy.extensions.logstats.LogStats']
- 2019-06-22 11:36:27 [scrapy.middleware] INFO: Enabled downloader middlewares:
- ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
- 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
- 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
- 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
- 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
- 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
- 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
- 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
- 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
- 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
- 'scrapy.downloadermiddlewares.stats.DownloaderStats']
- 2019-06-22 11:36:27 [scrapy.middleware] INFO: Enabled spider middlewares:
- ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
- 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
- 'scrapy.spidermiddlewares.referer.RefererMiddleware',
- 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
- 'scrapy.spidermiddlewares.depth.DepthMiddleware']
- 2019-06-22 11:36:27 [scrapy.middleware] INFO: Enabled item pipelines:
- ['collectSports.pipelines.sportslst.SportslstPipeline']
- 2019-06-22 11:36:27 [scrapy.core.engine] INFO: Spider opened
- 2019-06-22 11:36:27 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
- 2019-06-22 11:36:27 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
- 2019-06-22 11:36:28 [scrapy.core.downloader.tls] WARNING: Ignoring error while verifying certificate from host "205.201.4.177" (exception: ValueError('Invalid DNS-ID.'))
- 2019-06-22 11:38:03 [scrapy.extensions.logstats] INFO: Crawled 1 pages (at 1 pages/min), scraped 1 items (at 1 items/min)
- 2019-06-22 11:38:03 [scrapy.core.engine] INFO: Closing spider (finished)
- 2019-06-22 11:38:03 [scrapy.core.engine] ERROR: Scraper close failure
- Traceback (most recent call last):
- File "C:\ProgramData\Anaconda3\lib\site-packages\twisted\internet\defer.py", line 654, in _runCallbacks
- current.result = callback(current.result, *args, **kw)
- File "D:\workspace\sports_collect\collectSports\pipelines\sportslst.py", line 170, in close_spider
- self.client.close()
- AttributeError: 'SportslstPipeline' object has no attribute 'client'
- 2019-06-22 11:38:03 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
- {'downloader/request_bytes': 455,
- 'downloader/request_count': 1,
- 'downloader/request_method_count/GET': 1,
- 'downloader/response_bytes': 4730,
- 'downloader/response_count': 1,
- 'downloader/response_status_count/200': 1,
- 'finish_reason': 'finished',
- 'finish_time': datetime.datetime(2019, 6, 22, 3, 38, 3, 318581),
- 'item_scraped_count': 1,
- 'log_count/ERROR': 1,
- 'log_count/INFO': 10,
- 'log_count/WARNING': 1,
- 'response_received_count': 1,
- 'scheduler/dequeued': 1,
- 'scheduler/dequeued/memory': 1,
- 'scheduler/enqueued': 1,
- 'scheduler/enqueued/memory': 1,
- 'start_time': datetime.datetime(2019, 6, 22, 3, 36, 27, 790549)}
- 2019-06-22 11:38:03 [scrapy.core.engine] INFO: Spider closed (finished)
- 2019-06-22 11:39:28 [scrapy.utils.log] INFO: Scrapy 1.6.0 started (bot: SportsSpider)
- 2019-06-22 11:39:28 [scrapy.utils.log] INFO: Versions: lxml 4.3.2.0, libxml2 2.9.9, cssselect 1.0.3, parsel 1.5.1, w3lib 1.20.0, Twisted 19.2.1, Python 3.7.3 (default, Mar 27 2019, 17:13:21) [MSC v.1915 64 bit (AMD64)], pyOpenSSL 19.0.0 (OpenSSL 1.1.1b 26 Feb 2019), cryptography 2.6.1, Platform Windows-10-10.0.17763-SP0
- 2019-06-22 11:39:28 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'SportsSpider', 'LOG_FILE': 'SportsSpiderlog20190622.txt', 'LOG_LEVEL': 'INFO', 'NEWSPIDER_MODULE': 'collectSports.spiders', 'SPIDER_MODULES': ['collectSports.spiders'], 'USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 UBrowser/6.2.4098.3 Safari/537.36'}
- 2019-06-22 11:39:28 [scrapy.extensions.telnet] INFO: Telnet Password: 1bd2fb0ec3dc9450
- 2019-06-22 11:39:28 [scrapy.middleware] INFO: Enabled extensions:
- ['scrapy.extensions.corestats.CoreStats',
- 'scrapy.extensions.telnet.TelnetConsole',
- 'scrapy.extensions.logstats.LogStats']
- 2019-06-22 11:39:29 [scrapy.middleware] INFO: Enabled downloader middlewares:
- ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
- 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
- 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
- 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
- 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
- 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
- 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
- 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
- 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
- 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
- 'scrapy.downloadermiddlewares.stats.DownloaderStats']
- 2019-06-22 11:39:29 [scrapy.middleware] INFO: Enabled spider middlewares:
- ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
- 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
- 'scrapy.spidermiddlewares.referer.RefererMiddleware',
- 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
- 'scrapy.spidermiddlewares.depth.DepthMiddleware']
- 2019-06-22 11:39:29 [scrapy.middleware] INFO: Enabled item pipelines:
- ['collectSports.pipelines.sportslst.SportslstPipeline']
- 2019-06-22 11:39:29 [scrapy.core.engine] INFO: Spider opened
- 2019-06-22 11:39:29 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
- 2019-06-22 11:39:29 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
- 2019-06-22 11:39:29 [scrapy.core.downloader.tls] WARNING: Ignoring error while verifying certificate from host "205.201.4.177" (exception: ValueError('Invalid DNS-ID.'))
- 2019-06-22 11:40:22 [scrapy.utils.log] INFO: Scrapy 1.6.0 started (bot: SportsSpider)
- 2019-06-22 11:40:22 [scrapy.utils.log] INFO: Versions: lxml 4.3.2.0, libxml2 2.9.9, cssselect 1.0.3, parsel 1.5.1, w3lib 1.20.0, Twisted 19.2.1, Python 3.7.3 (default, Mar 27 2019, 17:13:21) [MSC v.1915 64 bit (AMD64)], pyOpenSSL 19.0.0 (OpenSSL 1.1.1b 26 Feb 2019), cryptography 2.6.1, Platform Windows-10-10.0.17763-SP0
- 2019-06-22 11:40:22 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'SportsSpider', 'LOG_FILE': 'SportsSpiderlog20190622.txt', 'LOG_LEVEL': 'INFO', 'NEWSPIDER_MODULE': 'collectSports.spiders', 'SPIDER_MODULES': ['collectSports.spiders'], 'USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 UBrowser/6.2.4098.3 Safari/537.36'}
- 2019-06-22 11:40:22 [scrapy.extensions.telnet] INFO: Telnet Password: 551a6dcf05de3a37
- 2019-06-22 11:40:22 [scrapy.middleware] INFO: Enabled extensions:
- ['scrapy.extensions.corestats.CoreStats',
- 'scrapy.extensions.telnet.TelnetConsole',
- 'scrapy.extensions.logstats.LogStats']
- 2019-06-22 11:40:22 [scrapy.middleware] INFO: Enabled downloader middlewares:
- ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
- 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
- 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
- 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
- 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
- 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
- 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
- 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
- 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
- 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
- 'scrapy.downloadermiddlewares.stats.DownloaderStats']
- 2019-06-22 11:40:22 [scrapy.middleware] INFO: Enabled spider middlewares:
- ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
- 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
- 'scrapy.spidermiddlewares.referer.RefererMiddleware',
- 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
- 'scrapy.spidermiddlewares.depth.DepthMiddleware']
- 2019-06-22 11:40:23 [scrapy.middleware] INFO: Enabled item pipelines:
- ['collectSports.pipelines.sportslst.SportslstPipeline']
- 2019-06-22 11:40:23 [scrapy.core.engine] INFO: Spider opened
- 2019-06-22 11:40:23 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
- 2019-06-22 11:40:23 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
- 2019-06-22 11:40:23 [scrapy.core.downloader.tls] WARNING: Ignoring error while verifying certificate from host "205.201.4.177" (exception: ValueError('Invalid DNS-ID.'))
- 2019-06-22 11:41:46 [scrapy.extensions.logstats] INFO: Crawled 1 pages (at 1 pages/min), scraped 1 items (at 1 items/min)
- 2019-06-22 11:41:46 [scrapy.core.engine] INFO: Closing spider (finished)
- 2019-06-22 11:41:46 [scrapy.core.engine] ERROR: Scraper close failure
- Traceback (most recent call last):
- File "C:\ProgramData\Anaconda3\lib\site-packages\twisted\internet\defer.py", line 654, in _runCallbacks
- current.result = callback(current.result, *args, **kw)
- File "D:\workspace\sports_collect\collectSports\pipelines\sportslst.py", line 170, in close_spider
- self.client.close()
- AttributeError: 'SportslstPipeline' object has no attribute 'client'
- 2019-06-22 11:41:46 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
- {'downloader/request_bytes': 490,
- 'downloader/request_count': 1,
- 'downloader/request_method_count/GET': 1,
- 'downloader/response_bytes': 19800,
- 'downloader/response_count': 1,
- 'downloader/response_status_count/200': 1,
- 'finish_reason': 'finished',
- 'finish_time': datetime.datetime(2019, 6, 22, 3, 41, 46, 856780),
- 'item_scraped_count': 1,
- 'log_count/ERROR': 1,
- 'log_count/INFO': 10,
- 'log_count/WARNING': 1,
- 'response_received_count': 1,
- 'scheduler/dequeued': 1,
- 'scheduler/dequeued/memory': 1,
- 'scheduler/enqueued': 1,
- 'scheduler/enqueued/memory': 1,
- 'start_time': datetime.datetime(2019, 6, 22, 3, 40, 23, 8292)}
- 2019-06-22 11:41:46 [scrapy.core.engine] INFO: Spider closed (finished)
|