Skip to content
Projects
Groups
Snippets
Help
This project
Loading...
Sign in / Register
Toggle navigation
T
toutiao
Overview
Overview
Details
Activity
Cycle Analytics
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
Issues
0
Issues
0
List
Board
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Charts
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Charts
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
zhiwei
toutiao
Commits
eb54f471
Commit
eb54f471
authored
Jan 16, 2019
by
[zhangzhiwei]
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
修改评论http访问方式
parent
cb6a0b84
Show whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
9 additions
and
8 deletions
+9
-8
src/main/java/com/zhiwei/toutiao/parse/TouTiaoCommentParse.java
+9
-8
No files found.
src/main/java/com/zhiwei/toutiao/parse/TouTiaoCommentParse.java
View file @
eb54f471
...
...
@@ -13,6 +13,8 @@ import org.apache.logging.log4j.Logger;
import
com.alibaba.fastjson.JSONArray
;
import
com.alibaba.fastjson.JSONObject
;
import
com.zhiwei.crawler.core.HttpBoot
;
import
com.zhiwei.crawler.core.RequestUtils
;
import
com.zhiwei.tools.httpclient.HttpClientTemplateOK
;
import
com.zhiwei.tools.tools.ZhiWeiTools
;
import
com.zhiwei.toutiao.bean.TouTiaoComment
;
...
...
@@ -27,7 +29,7 @@ import com.zhiwei.toutiao.util.Tools;
public
class
TouTiaoCommentParse
{
private
static
Logger
logger
=
LogManager
.
getLogger
(
TouTiaoCommentParse
.
class
);
private
static
HttpBoot
httpBoot
=
new
HttpBoot
();
/**
*
...
...
@@ -135,7 +137,7 @@ public class TouTiaoCommentParse {
String
urlNew
=
"http://www.toutiao.com/api/comment/list/?group_id="
+
group_id
+
"&item_id=0&count=20&offset=0"
;
//设置头信息
Map
<
String
,
String
>
headerMap
=
Tools
.
getTouTiaoHeader
();
String
htmlBody
=
HttpClientTemplateOK
.
get
(
urlNew
,
proxy
,
headerMap
);
String
htmlBody
=
httpBoot
.
syncCall
(
RequestUtils
.
wrapGet
(
urlNew
,
headerMap
),
proxy
).
body
().
string
(
);
if
(
htmlBody
!=
null
)
{
...
...
@@ -172,7 +174,7 @@ public class TouTiaoCommentParse {
try
{
//设置头信息
Map
<
String
,
String
>
headerMap
=
Tools
.
getTouTiaoHeader
();
String
htmlBody
=
HttpClientTemplateOK
.
get
(
url
,
proxy
,
headerMap
);
String
htmlBody
=
httpBoot
.
syncCall
(
RequestUtils
.
wrapGet
(
url
,
headerMap
),
proxy
).
body
().
string
(
);
if
(
htmlBody
!=
null
&&
htmlBody
.
contains
(
"commentInfo"
))
{
try
{
...
...
@@ -199,12 +201,12 @@ public class TouTiaoCommentParse {
* @param @return 设定文件
* @return int 返回类型
*/
public
static
int
find
CommentCountByNew
Proxy
(
String
url
,
Proxy
proxy
)
public
static
int
find
NewCommentCountBy
Proxy
(
String
url
,
Proxy
proxy
)
{
try
{
//设置头信息
Map
<
String
,
String
>
headerMap
=
Tools
.
getTouTiaoHeader
();
String
htmlBody
=
HttpClientTemplateOK
.
get
(
url
,
proxy
,
headerMap
);
String
htmlBody
=
httpBoot
.
syncCall
(
RequestUtils
.
wrapGet
(
url
,
headerMap
),
proxy
).
body
().
string
(
);
if
(
htmlBody
!=
null
&&
htmlBody
.
contains
(
"commentInfo"
))
{
try
{
...
...
@@ -236,8 +238,7 @@ public class TouTiaoCommentParse {
String
urlNew
=
"http://www.toutiao.com/api/comment/list/?group_id="
+
group_id
+
"&item_id=0&count=20&offset=0"
;
//设置头信息
Map
<
String
,
String
>
headerMap
=
Tools
.
getTouTiaoHeader
();
String
htmlBody
=
HttpClientTemplateOK
.
get
(
urlNew
,
proxy
,
headerMap
);
String
htmlBody
=
httpBoot
.
syncCall
(
RequestUtils
.
wrapGet
(
urlNew
,
headerMap
),
proxy
).
body
().
string
();
if
(
htmlBody
!=
null
)
{
try
{
...
...
@@ -300,7 +301,7 @@ public class TouTiaoCommentParse {
String
groupId
=
null
;
Map
<
String
,
String
>
headerMap
=
Tools
.
getTouTiaoHeader
();
try
{
String
htmlBody
=
HttpClientTemplateOK
.
get
(
url
,
proxy
,
headerMap
);
String
htmlBody
=
httpBoot
.
syncCall
(
RequestUtils
.
wrapGet
(
url
,
headerMap
),
proxy
).
body
().
string
(
);
if
(
htmlBody
!=
null
)
{
if
(
htmlBody
.
contains
(
"groupId"
))
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment