python - 關(guān)于Scrapy中TwistedPipline報(bào)錯
問題描述
> python3.5 / pycharm
crawlspider 返回的 item 是絕對沒有問題的，接上TwistedPipline就報(bào)錯 請哪位老師幫忙分析下報(bào)錯代碼
# TwistedPiplineclass MysqlTwistedPipline(object): def __init__(self, dbpool):self.dbpool = dbpool @classmethod def from_settings(cls, settings):dbparms = dict( host = settings['MYSQL_HOST'], db = settings['MYSQL_DBNAME'], user = settings['MYSQL_USER'], passwd = settings['MYSQL_PASSWORD'], charset=’utf8’, cursorclass=MySQLdb.cursors.DictCursor, use_unicode=True,)dbpool = adbapi.ConnectionPool('MySQLdb', **dbparms)return cls(dbpool) def process_item(self, item, spider):query = self.dbpool.runInteraction(self.do_insert, item)query.addErrback(self.handle_error, item, spider) def handle_error(self, failure, item, spider):print (failure) def do_insert(self, cursor, item):insert_sql, params = item.insert_values()print (insert_sql, params)cursor.execute(insert_sql, params)return item
Error code
2017-05-01 00:06:16 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.lagou.com/jobs/2108656.html> (referer: https://www.lagou.com/zhaopin/Python/)2017-05-01 00:06:16 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.lagou.com/jobs/2108656.html>None[Failure instance: Traceback: <class ’TypeError’>: connect() argument 3 must be str, not NoneD:Python35libthreading.py:914:_bootstrap_innerD:Python35libthreading.py:862:runD:Python35libsite-packagestwisted_threads_threadworker.py:46:workD:Python35libsite-packagestwisted_threads_team.py:190:doWork--- <exception caught here> ---D:Python35libsite-packagestwistedpythonthreadpool.py:250:inContextD:Python35libsite-packagestwistedpythonthreadpool.py:266:<lambda>D:Python35libsite-packagestwistedpythoncontext.py:122:callWithContextD:Python35libsite-packagestwistedpythoncontext.py:85:callWithContextD:Python35libsite-packagestwistedenterpriseadbapi.py:464:_runInteractionD:Python35libsite-packagestwistedenterpriseadbapi.py:36:__init__D:Python35libsite-packagestwistedenterpriseadbapi.py:76:reconnectD:Python35libsite-packagestwistedenterpriseadbapi.py:431:connectD:Python35libsite-packagesMySQLdb__init__.py:86:ConnectD:Python35libsite-packagesMySQLdbconnections.py:204:__init__]
問題解答
回答1：從提示看出,你的配置(settings)的時候有1個參數(shù)是None,而不是字符串
相關(guān)文章：
1. android - weex 項(xiàng)目createInstanceReferenceError: Vue is not defined2. android - 哪位大神知道java后臺的api接口的對象傳到前端后輸入日期報(bào)錯，是什么情況？求大神指點(diǎn)3. PHPExcel表格導(dǎo)入數(shù)據(jù)庫怎么導(dǎo)入4. pdo 寫入到數(shù)據(jù)庫的內(nèi)容為中文的時候?qū)懭雭y碼5. javascript - 如圖，百度首頁，查看源代碼為什么什么都沒有？6. vue2.0+webpack 如何使用bootstrap？7. PHP類封裝的插入數(shù)據(jù)，總是插入不成功，返回false；8. mac連接阿里云docker集群，已經(jīng)卡了2天了，求問？9. 微信渠道二維碼怎么使用？10. html5 - html元素select下拉列表在原生app里面為什么不能彈框顯示？
網(wǎng)公網(wǎng)安備