Skip to content
Projects
Groups
Snippets
Help
This project
Loading...
Sign in / Register
Toggle navigation
B
brandkbs2
Overview
Overview
Details
Activity
Cycle Analytics
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
Issues
0
Issues
0
List
Board
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Charts
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Charts
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
shenjunjie
brandkbs2
Commits
5c6c6d0f
Commit
5c6c6d0f
authored
Aug 16, 2023
by
shenjunjie
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
天级定时任务es同步添加重试
parent
a320e10f
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
14 additions
and
6 deletions
+14
-6
src/main/java/com/zhiwei/brandkbs2/service/impl/TaskServiceImpl.java
+14
-6
No files found.
src/main/java/com/zhiwei/brandkbs2/service/impl/TaskServiceImpl.java
View file @
5c6c6d0f
...
...
@@ -18,6 +18,7 @@ import org.apache.commons.collections4.ListUtils;
import
org.apache.commons.lang3.tuple.Pair
;
import
org.apache.logging.log4j.LogManager
;
import
org.apache.logging.log4j.Logger
;
import
org.elasticsearch.ElasticsearchStatusException
;
import
org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor
;
import
org.springframework.stereotype.Service
;
...
...
@@ -125,12 +126,19 @@ public class TaskServiceImpl implements TaskService {
List
<
ChannelRecord
>
channelRecords
=
ChannelRecord
.
createChannelRecords
(
timeMinMax
[
0
],
timeMinMax
[
1
],
channelIndexRecordMap
);
channelEsDao
.
upsertChannelRecord
(
channelRecords
);
// 同步ES-channelCopy,区分insertList和updateList
ListUtils
.
partition
(
insertList
,
1000
).
forEach
(
list
->
{
channelEsDao
.
batchInsert
(
list
.
stream
().
map
(
Channel:
:
createChannelCopyMap
).
collect
(
Collectors
.
toList
()));
});
ListUtils
.
partition
(
updateList
,
1000
).
forEach
(
list
->
{
channelEsDao
.
batchInsert
(
list
.
stream
().
map
(
Channel:
:
createChannelCopyMap
).
collect
(
Collectors
.
toList
()));
});
for
(
int
i
=
1
;
i
<=
3
;
i
++)
{
try
{
ListUtils
.
partition
(
insertList
,
1000
).
forEach
(
list
->
{
channelEsDao
.
batchInsert
(
list
.
stream
().
map
(
Channel:
:
createChannelCopyMap
).
collect
(
Collectors
.
toList
()));
});
ListUtils
.
partition
(
updateList
,
1000
).
forEach
(
list
->
{
channelEsDao
.
batchInsert
(
list
.
stream
().
map
(
Channel:
:
createChannelCopyMap
).
collect
(
Collectors
.
toList
()));
});
break
;
}
catch
(
ElasticsearchStatusException
e
)
{
log
.
info
(
"渠道统计-es入库解析异常,正在重试第{}次"
,
i
);
}
}
log
.
info
(
"渠道统计-渠道记录-统计结束"
);
}
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment