Skip to content
GitLab
Menu
Projects
Groups
Snippets
/
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
Felix Seibert
xtreemfs_client
Commits
1fba7a40
Commit
1fba7a40
authored
Mar 20, 2018
by
Felix Seibert
Browse files
adding progress tracker
parent
c97a6b13
Changes
1
Hide whitespace changes
Inline
Side-by-side
xtreemfs_client/div_util.py
View file @
1fba7a40
...
...
@@ -51,6 +51,8 @@ execute list of commands in parallel, return list of executions returned with an
def
run_commands
(
commands
,
max_processes
=
200
,
print_errors
=
True
):
running_processes
=
set
()
errored_processes
=
[]
num_finished
=
0
num_total
=
len
(
commands
)
for
command
in
commands
:
started_process
=
subprocess
.
Popen
(
command
,
stderr
=
subprocess
.
PIPE
,
stdout
=
subprocess
.
PIPE
,
shell
=
True
)
running_processes
.
add
(
started_process
)
...
...
@@ -59,6 +61,7 @@ def run_commands(commands, max_processes=200, print_errors=True):
difference
=
set
()
for
running_process
in
running_processes
:
if
running_process
.
poll
()
is
not
None
:
num_finished
+=
1
difference
.
add
(
running_process
)
finished_process
=
(
running_process
.
args
,
running_process
.
communicate
(),
...
...
@@ -67,6 +70,7 @@ def run_commands(commands, max_processes=200, print_errors=True):
errored_processes
.
append
(
finished_process
)
if
print_errors
:
print_error
(
finished_process
)
print
(
"progress: "
+
str
(
num_finished
)
+
"/"
+
str
(
num_total
))
running_processes
=
running_processes
.
difference
(
difference
)
...
...
Write
Preview
Supports
Markdown
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment