Skip to content
Projects
Groups
Snippets
Help
This project
Loading...
Sign in / Register
Toggle navigation
S
searchhotcrawler
Overview
Overview
Details
Activity
Cycle Analytics
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
Issues
0
Issues
0
List
Board
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Charts
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Charts
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
zhiwei
searchhotcrawler
Commits
74064b44
Commit
74064b44
authored
Sep 16, 2021
by
chenweitao
Browse files
Options
Browse Files
Download
Plain Diff
Merge remote-tracking branch 'origin/working' into working
parents
9c03b231
29ea7f00
Show whitespace changes
Inline
Side-by-side
Showing
3 changed files
with
19 additions
and
1 deletions
+19
-1
src/main/java/com/zhiwei/searchhotcrawler/bean/HotSearchList.java
+4
-0
src/main/java/com/zhiwei/searchhotcrawler/crawler/WeiboHotSearchCrawler.java
+14
-1
src/main/java/com/zhiwei/searchhotcrawler/dao/HotSearchCacheDAO.java
+1
-0
No files found.
src/main/java/com/zhiwei/searchhotcrawler/bean/HotSearchList.java
View file @
74064b44
...
@@ -73,6 +73,10 @@ public class HotSearchList implements Serializable{
...
@@ -73,6 +73,10 @@ public class HotSearchList implements Serializable{
* 热搜类型
* 热搜类型
*/
*/
private
String
icon
;
private
String
icon
;
/**
* 热度标签
*/
private
String
heatLabel
;
/**
/**
* icon地址
* icon地址
...
...
src/main/java/com/zhiwei/searchhotcrawler/crawler/WeiboHotSearchCrawler.java
View file @
74064b44
...
@@ -165,7 +165,19 @@ public class WeiboHotSearchCrawler {
...
@@ -165,7 +165,19 @@ public class WeiboHotSearchCrawler {
for
(
int
j
=
0
;
j
<
cardGroup
.
size
();
j
++)
{
for
(
int
j
=
0
;
j
<
cardGroup
.
size
();
j
++)
{
JSONObject
cardInfo
=
cardGroup
.
getJSONObject
(
j
);
JSONObject
cardInfo
=
cardGroup
.
getJSONObject
(
j
);
String
name
=
cardInfo
.
getString
(
"desc"
);
String
name
=
cardInfo
.
getString
(
"desc"
);
long
hotCount
=
cardInfo
.
getLongValue
(
"desc_extr"
);
String
desc_extr
=
cardInfo
.
getString
(
"desc_extr"
);
String
heatLabel
=
null
;
Long
hotCount
=
null
;
if
(
Objects
.
nonNull
(
desc_extr
)){
String
[]
split
=
desc_extr
.
split
(
" "
);
if
(
split
.
length
>
1
){
heatLabel
=
split
[
0
].
trim
();
hotCount
=
Long
.
valueOf
(
split
[
1
].
trim
());
}
else
{
hotCount
=
cardInfo
.
getLongValue
(
"desc_extr"
);
}
}
String
iconUrl
=
cardInfo
.
getString
(
"icon"
);
String
iconUrl
=
cardInfo
.
getString
(
"icon"
);
String
icon
=
null
;
String
icon
=
null
;
if
(
StringUtils
.
isNotBlank
(
iconUrl
))
{
if
(
StringUtils
.
isNotBlank
(
iconUrl
))
{
...
@@ -174,6 +186,7 @@ public class WeiboHotSearchCrawler {
...
@@ -174,6 +186,7 @@ public class WeiboHotSearchCrawler {
// String id = "http://s.weibo.com/weibo/" + URLCodeUtil.getURLEncode(name, "utf-8") + "&Refer=top";
// String id = "http://s.weibo.com/weibo/" + URLCodeUtil.getURLEncode(name, "utf-8") + "&Refer=top";
String
id
=
cardInfo
.
getString
(
"scheme"
);
String
id
=
cardInfo
.
getString
(
"scheme"
);
HotSearchList
hotSearch
=
new
HotSearchList
(
id
,
name
,
hotCount
,
hot
,
rank
,
HotSearchType
.
微博热搜
.
name
(),
icon
,
date
);
HotSearchList
hotSearch
=
new
HotSearchList
(
id
,
name
,
hotCount
,
hot
,
rank
,
HotSearchType
.
微博热搜
.
name
(),
icon
,
date
);
hotSearch
.
setHeatLabel
(
heatLabel
);
if
(
Objects
.
nonNull
(
iconUrl
)){
hotSearch
.
setIconUrl
(
iconUrl
);}
if
(
Objects
.
nonNull
(
iconUrl
)){
hotSearch
.
setIconUrl
(
iconUrl
);}
result
.
add
(
hotSearch
);
result
.
add
(
hotSearch
);
rank
++;
rank
++;
...
...
src/main/java/com/zhiwei/searchhotcrawler/dao/HotSearchCacheDAO.java
View file @
74064b44
...
@@ -52,6 +52,7 @@ public class HotSearchCacheDAO {
...
@@ -52,6 +52,7 @@ public class HotSearchCacheDAO {
}
}
if
(
"微博热搜"
.
equals
(
hotSearch
.
getType
())){
if
(
"微博热搜"
.
equals
(
hotSearch
.
getType
())){
document
.
put
(
"iconUrl"
,
hotSearch
.
getIconUrl
());
document
.
put
(
"iconUrl"
,
hotSearch
.
getIconUrl
());
document
.
put
(
"heatLabel"
,
hotSearch
.
getHeatLabel
());
}
}
if
(
"微博娱乐榜"
.
equals
(
hotSearch
.
getType
())){
if
(
"微博娱乐榜"
.
equals
(
hotSearch
.
getType
())){
document
.
put
(
"iconUrl"
,
hotSearch
.
getIconUrl
());
document
.
put
(
"iconUrl"
,
hotSearch
.
getIconUrl
());
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment