java에서 mysql 대량 삽입을 실행하는 몇 가지 방법 및 사용 시

3304 단어 mysql대량 삽입
방법 1:
Java code

conn = DriverManager.getConnection(JDBC_URL, JDBC_USER, JDBC_PASS);
        pstmt = conn
                .prepareStatement("insert into loadtest (id, data) values (?, ?)");
        for (int i = 1; i <= COUNT; i++) {
            pstmt.clearParameters();
            pstmt.setInt(1, i);
            pstmt.setString(2, DATA);
            pstmt.execute();
        }
MyISAM: 246.6초, InnoDB: 360.2초
방법 2: 트랜잭션 사용, 자동commit 없음
Java code

conn = DriverManager.getConnection(JDBC_URL, JDBC_USER, JDBC_PASS);
        conn.setAutoCommit(false);
        pstmt = conn
                .prepareStatement("insert into loadtest (id, data) values (?, ?)");
        for (int i = 1; i <= COUNT; i++) {
            pstmt.clearParameters();
            pstmt.setInt(1, i);
            pstmt.setString(2, DATA);
            pstmt.execute();
            if (i % COMMIT_SIZE == 0) {
                conn.commit();
            }
        }
        conn.commit();
InnoDB: 31.5초
방법 3: executeBatch
Java code

conn = DriverManager.getConnection(JDBC_URL
                + "?rewriteBatchedStatements=true", JDBC_USER, JDBC_PASS);
        conn.setAutoCommit(false);
        pstmt = conn
                .prepareStatement("insert into loadtest (id, data) values (?, ?)");
        for (int i = 1; i <= COUNT; i += BATCH_SIZE) {
            pstmt.clearBatch();
            for (int j = 0; j < BATCH_SIZE; j++) {
                pstmt.setInt(1, i + j);
                pstmt.setString(2, DATA);
                pstmt.addBatch();
            }
            pstmt.executeBatch();
            if ((i + BATCH_SIZE - 1) % COMMIT_SIZE == 0) {
                conn.commit();
            }
        }
        conn.commit();
InnoDB: 5.2초
위의 사용 시 반드시 1) rewriteBatchedStatements=true2) useServerPrepStmts=true
방법 4: LOAD 후 COMMIT
Java code

conn = DriverManager.getConnection(JDBC_URL, JDBC_USER, JDBC_PASS);
        conn.setAutoCommit(false);
        pstmt = conn.prepareStatement("load data local infile '' "
                + "into table loadtest fields terminated by ','");
        StringBuilder sb = new StringBuilder();
        for (int i = 1; i <= COUNT; i++) {
            sb.append(i + "," + DATA + "
");
            if (i % COMMIT_SIZE == 0) {
                InputStream is = new ByteArrayInputStream(sb.toString()
                        .getBytes());
                ((com.mysql.jdbc.Statement) pstmt)
                        .setLocalInfileInputStream(is);
                pstmt.execute();
                conn.commit();
                sb.setLength(0);
            }
        }
        InputStream is = new ByteArrayInputStream(sb.toString().getBytes());
        ((com.mysql.jdbc.Statement) pstmt).setLocalInfileInputStream(is);
        pstmt.execute();
        conn.commit();

좋은 웹페이지 즐겨찾기