Skip to content
Projects
Groups
Snippets
Help
This project
Loading...
Sign in / Register
Toggle navigation
S
source_forward
Overview
Overview
Details
Activity
Cycle Analytics
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
Issues
0
Issues
0
List
Board
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Charts
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Charts
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
zhiwei
source_forward
Commits
dfc68c42
Commit
dfc68c42
authored
May 07, 2020
by
cwy
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
增加懂车帝 自媒体号获取
parent
132b70cb
Show whitespace changes
Inline
Side-by-side
Showing
4 changed files
with
8 additions
and
8 deletions
+8
-8
src/main/java/com/zhiwei/source_forward/crawler/MediaSelfSourceCrawler.java
+0
-1
src/main/java/com/zhiwei/source_forward/run/MediaSelfSource.java
+1
-2
src/main/java/com/zhiwei/source_forward/run/URLLive.java
+1
-5
src/main/java/com/zhiwei/source_forward/util/MatchSource.java
+6
-0
No files found.
src/main/java/com/zhiwei/source_forward/crawler/MediaSelfSourceCrawler.java
View file @
dfc68c42
...
...
@@ -5,7 +5,6 @@ import java.util.List;
import
java.util.Map
;
import
java.util.Objects
;
import
org.apache.commons.lang3.StringUtils
;
import
org.apache.logging.log4j.LogManager
;
import
org.apache.logging.log4j.Logger
;
import
org.jsoup.Jsoup
;
...
...
src/main/java/com/zhiwei/source_forward/run/MediaSelfSource.java
View file @
dfc68c42
...
...
@@ -4,7 +4,6 @@ import java.util.ArrayList;
import
java.util.Collections
;
import
java.util.List
;
import
com.zhiwei.proxy.config.SimpleConfig
;
import
com.zhiwei.source_forward.util.ProxyInit
;
import
org.apache.logging.log4j.LogManager
;
import
org.apache.logging.log4j.Logger
;
...
...
@@ -33,7 +32,7 @@ public class MediaSelfSource {
ProxyInit
.
initProxy
();
List
<
String
>
urlList
=
new
ArrayList
<>();
urlList
.
add
(
"https://www.
tuicool.com/articles/nIfmu2B
"
);
urlList
.
add
(
"https://www.
dcdapp.com/article/6819085953756299789
"
);
List
<
MediaSelfSourceBean
>
u
=
MediaSelfSource
.
getMediaSelfSource
(
urlList
);
for
(
MediaSelfSourceBean
b
:
u
)
{
System
.
out
.
println
(
b
.
toString
());
...
...
src/main/java/com/zhiwei/source_forward/run/URLLive.java
View file @
dfc68c42
...
...
@@ -6,14 +6,13 @@ import java.util.List;
import
java.util.Map
;
import
java.util.Map.Entry
;
import
com.zhiwei.source_forward.util.ProxyInit
;
import
org.apache.logging.log4j.LogManager
;
import
org.apache.logging.log4j.Logger
;
import
com.zhiwei.crawler.core.HttpBoot
;
import
com.zhiwei.source_forward.bean.UrlLiveBean
;
import
com.zhiwei.source_forward.bean.UrlLiveBean.Attribution
;
import
com.zhiwei.source_forward.crawler.UrlLiveCrawler
;
import
com.zhiwei.source_forward.util.ProxyInit
;
import
com.zhiwei.source_forward.util.UrlLiveDataCallback
;
/**
...
...
@@ -26,8 +25,6 @@ public class URLLive {
private
static
Logger
logger
=
LogManager
.
getLogger
(
URLLive
.
class
);
private
static
HttpBoot
httpBoot
=
new
HttpBoot
.
Builder
().
build
();
/**
* @Title: verificationURLLive
* @author hero
...
...
@@ -42,7 +39,6 @@ public class URLLive {
for
(
Entry
<
String
,
Map
<
String
,
Object
>>
entry
:
dataMap
.
entrySet
()){
urlList
.
add
(
entry
.
getKey
());
}
System
.
out
.
println
(
urlList
.
size
());
//验证数据是否已删除
List
<
UrlLiveBean
>
dataList
=
UrlLiveCrawlerThread
.
getUrlLiveCrawle
(
urlList
);
for
(
UrlLiveBean
ub
:
dataList
){
...
...
src/main/java/com/zhiwei/source_forward/util/MatchSource.java
View file @
dfc68c42
...
...
@@ -372,6 +372,12 @@ public class MatchSource {
source
=
"搜狐-"
+
source
;
}
}
}
else
if
(
url
.
contains
(
"dcdapp.com/article"
)){
// 懂车帝
source
=
html
.
split
(
"screen_name\":\""
)[
1
].
split
(
"\"}"
)[
0
].
trim
();
if
(
source
!=
null
&&
source
.
length
()>
1
){
source
=
"懂车帝-"
+
source
;
}
}
else
if
(
url
.
contains
(
"tznew.58.com"
)){
source
=
JSONObject
.
parseObject
(
html
).
getJSONObject
(
"result"
).
getString
(
"author"
);
if
(
source
!=
null
&&
source
.
length
()>
1
){
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment