query_builder

crawlib.pipeline.rds.query_builder.finished(finished_status, update_interval, table, status_column, edit_at_column)

Create text sql statement query for sqlalchemy that getting all finished task.

Parameters:

• finished_status – int, status code that greater or equal than this will be considered as finished.
• update_interval – int, the record will be updated every x seconds.

Returns:

sqlalchemy text sql statement.

中文文档

状态码大于某个值, 并且, 更新时间在最近一段时间以内.

crawlib.pipeline.rds.query_builder.unfinished(finished_status, update_interval, table, status_column, edit_at_column)

Create text sql statement query for sqlalchemy that getting all unfinished task.

Parameters:

• finished_status – int, status code that less than this will be considered as unfinished.
• update_interval – int, the record will be updated every x seconds.

Returns:

sqlalchemy text sql statement.

中文文档

状态码小于某个值, 或者, 现在距离更新时间已经超过一定阈值.