Python의 병렬 처리 정보

Python의 concurrent나는futures와multiprocessing의 행동을 테스트해 보았기 때문에 필기 대신 투고했다.
목적은 Python의 규격을 확인하는 것이 아니라 시종일관'어떻게 행동하는가'입니다.

CPU 스레드 수 및 매개변수 max_직원 정보

max_작업자와 프로세서의 스레드 수를 일치시켜야 합니까?

이런 의문이 갑자기 들끓어 검증이 이뤄졌다.
특별한 대조 실험과 통계학 검증을 하지 않았다.양해해 주십시오.

환경 확인

CPU core i7 6800k (6core,12thread base clock=3.40GHz)

Memory 32GB(DDR4-2133)

OS Windows 10 Pro

인증 코드 1

concurrent-futures.py

from concurrent import futures
from time import time as t


def primes(x):
    primelist_ = [2]
    for i in range(3, x + 1):
        for j in primelist_:
            if j ** 2 > i:
                primelist_.append(i)
                break
            elif i % j == 0:
                break
    return primelist_

if __name__ == "__main__":

    list_ = [1000] * 60
    for i in range(1,61):
        s = t()
        with futures.ProcessPoolExecutor(max_workers=i) as executor:
            executor.map(primes,list_)
        e = t()
        print("{0} workers time is {1}".format(i,e-s))

실행 결과 1

1 workers time is 0.5020229816436768
2 workers time is 0.352001428604126
3 workers time is 0.36597752571105957
4 workers time is 0.4050002098083496
5 workers time is 0.5020010471343994
6 workers time is 0.600996732711792
7 workers time is 0.8990006446838379
8 workers time is 1.0159986019134521
9 workers time is 1.1840019226074219
10 workers time is 1.265998363494873
11 workers time is 1.3460001945495605
12 workers time is 1.5809991359710693
13 workers time is 1.7050011157989502
14 workers time is 1.8249998092651367
15 workers time is 1.9100017547607422
16 workers time is 2.215999126434326
17 workers time is 2.0129990577697754
18 workers time is 1.860001564025879
19 workers time is 1.98099946975708
20 workers time is 2.031001091003418
21 workers time is 2.225998878479004
22 workers time is 2.263998508453369
23 workers time is 3.3330018520355225
24 workers time is 3.2790024280548096
25 workers time is 3.2919981479644775
26 workers time is 3.398635149002075
27 workers time is 3.5269999504089355
28 workers time is 3.714276075363159
29 workers time is 3.823046922683716
30 workers time is 3.9260809421539307
31 workers time is 4.0271430015563965
32 workers time is 4.247858047485352
33 workers time is 4.3211283683776855
34 workers time is 4.448809385299683
35 workers time is 4.6109983921051025
36 workers time is 5.038041353225708
37 workers time is 4.97698187828064
38 workers time is 5.122000694274902
39 workers time is 5.114000082015991
40 workers time is 5.294032096862793
41 workers time is 5.48790717124939
42 workers time is 5.696187257766724
43 workers time is 5.781583309173584
44 workers time is 5.892255783081055
45 workers time is 6.041832447052002
46 workers time is 6.1898744106292725
47 workers time is 5.498929738998413
48 workers time is 1.6420001983642578
49 workers time is 3.067247152328491
50 workers time is 2.9033234119415283
51 workers time is 6.299036264419556
52 workers time is 7.5803093910217285
53 workers time is 7.545795917510986
54 workers time is 7.648757219314575
55 workers time is 7.437737703323364
56 workers time is 7.803879022598267
57 workers time is 7.753534317016602
58 workers time is 8.295880317687988
59 workers time is 8.039872407913208
60 workers time is 8.545195579528809

도표에는 이런 느낌이 있다.

결과 1

많이 한다고 해서 좋은 것도 아니고,thread 수에 맞추면 좋은 것도 아니다.
죄송합니다. 다시 한 번 검증해 보겠습니다.

인증 코드 2

multiprocessing.py

from multiprocessing import Pool
from time import time as t


def primes(x=1000000000):
    primelist_ = [2]
    for i in range(3, x + 1):
        for j in primelist_:
            if j ** 2 > i:
                primelist_.append(i)
                break
            elif i % j == 0:
                break
    return primelist_


if __name__ == "__main__":
    list_ = [1000] * 60
    for i in range(1,61):
        p = Pool(i)
        s = t()
        p.map(primes, list_)
        e = t()
        print("{0} workers time is {1}".format(i, e - s))

실행 결과 2

1 workers time is 0.2633683681488037
2 workers time is 0.22099924087524414
3 workers time is 0.19600248336791992
4 workers time is 0.18899846076965332
5 workers time is 0.194000244140625
6 workers time is 0.20499968528747559
7 workers time is 0.22199773788452148
8 workers time is 0.22899842262268066
9 workers time is 0.25099897384643555
10 workers time is 0.27899861335754395
11 workers time is 0.2799994945526123
12 workers time is 0.29799890518188477
13 workers time is 0.3249998092651367
14 workers time is 0.328000545501709
15 workers time is 0.3409996032714844
16 workers time is 0.37000060081481934
17 workers time is 0.38399839401245117
18 workers time is 0.354999303817749
19 workers time is 0.17999982833862305
20 workers time is 0.453000545501709
21 workers time is 0.3829984664916992
22 workers time is 1.06699800491333
23 workers time is 4.132883548736572
24 workers time is 2.9350016117095947
25 workers time is 3.341001033782959
26 workers time is 3.19700026512146
27 workers time is 3.712002754211426
28 workers time is 3.354022741317749
29 workers time is 4.1504881381988525
30 workers time is 0.10300064086914062
31 workers time is 8.025314092636108
32 workers time is 4.032264471054077
33 workers time is 1.3879990577697754
34 workers time is 0.38199782371520996
35 workers time is 0.8919985294342041
36 workers time is 0.44299936294555664
37 workers time is 0.9930007457733154
38 workers time is 0.5849990844726562
39 workers time is 1.2019994258880615
40 workers time is 3.2748472690582275
41 workers time is 8.506859064102173
42 workers time is 5.890998840332031
43 workers time is 6.897999286651611
44 workers time is 0.09200048446655273
45 workers time is 47.79252886772156
46 workers time is 107.53967475891113
47 workers time is 85.88229584693909
48 workers time is 82.07777094841003
49 workers time is 86.7360258102417
50 workers time is 89.32221388816833
51 workers time is 87.51184272766113
52 workers time is 65.91199898719788
53 workers time is 120.46363306045532
54 workers time is 79.80397748947144
55 workers time is 55.082069873809814
56 workers time is 84.51597738265991
57 workers time is 114.39745855331421
58 workers time is 75.04559135437012
59 workers time is 109.53564620018005
60 workers time is 81.29796814918518

도표에는 이런 느낌이 있다.

이러면 보기 싫으니까 1-24로 한정해 봐요.

결과 2

multiprocessing을 사용하는 상황에서 CPU의thread 수 아래에 머무르는 것은 원형인 것 같습니다.