[TypeScript] MediaRecorder로 MediaStream 저장

60547 단어 typescript aspnetcore

소개

MediaRecorder로 비디오와 오디오를 저장해 보겠습니다.
WebRTC를 시도하기 위해 만들었을 때 프로젝트를 사용합니다.

기본 프로젝트

Index.cshtml

<!DOCTYPE html>
<html lang="en">
    <head>
        <title>Hello WebRTC</title>
        <meta charset="utf-8">
    </head>
    <body>
...
        <div id="webrtc_sample_area">
...
            <video id="local_video" muted>Video stream not available.</video>
            <video id="received_video" autoplay>Video stream not available.</video>
        </div>
        <div>
            <button onclick="Page.switchFrame()">Frame</button>
            <button onclick="Page.startRecording()">Start</button>
            <button onclick="Page.stopRecording()">Stop</button>            
        </div>
        <canvas id="picture_canvas"></canvas>
        <a id="download_target"></a>
        <script src="js/main.js"></script>
    </body>
</html>

main.page.ts

import { VideoRecorder } from "./video-recorder";
import { WebRtcController } from "./webrtc-controller";
...
let rtcSample = new WebRtcController();
let videoRecorder: VideoRecorder;
...
export function startRecording(): void {
    videoRecorder.startRecording();
}
export function stopRecording(): void {
    videoRecorder.stopRecording();
}
export function switchFrame(): void {
    videoRecorder.updateCanvasSize();
    videoRecorder.switchFrame();
}
...
function init(){
    rtcSample = new WebRtcController();
    rtcSample.initVideo();
    videoRecorder = new VideoRecorder();
}
init();

webrtc-controller.ts

...
export class WebRtcController {
...
  public initVideo(){
      const localVideo = document.getElementById("local_video") as HTMLVideoElement;
      let streaming = false;
      // after being UserMedia available, set Video element's size.
      localVideo.addEventListener("canplay", () => {
          if (streaming === false) {
            const width = 320;
            const height = localVideo.videoHeight / (localVideo.videoWidth/width);          
            localVideo.setAttribute("width", width.toString());
            localVideo.setAttribute("height", height.toString());
            streaming = true;
          }
        }, false);
      navigator.mediaDevices.getUserMedia({ video: true, audio: true })
        .then(stream => {
            this.webcamStream = stream;
            localVideo.srcObject = stream;
            localVideo.play();
            streaming = true;
        })
        .catch(err => console.error(`An error occurred: ${err}`));
  }
...

비디오 및 오디오 저장

비디오와 오디오를 저장할 수 있습니다

video-recorder.ts

export class VideoRecorder {
    private recorder: MediaRecorder|null = null;
    public startRecording() {
        const localVideo = document.getElementById("local_video") as HTMLVideoElement;
        const localVideoStream = this.getVideoStream(this.localVideo);
        if(localVideoStream != null) {
            this.recorder = new MediaRecorder(localVideoStream, this.getMimeType());
            this.recorder.ondataavailable = (ev) => this.saveRecordedVideo(ev);
            this.recorder.start();
        }
    }
    public stopRecording() {
        this.recorder?.stop();
    }
    private getVideoStream(targetElement: HTMLVideoElement): MediaStream|null {
        if(targetElement.srcObject != null &&
            "getTracks" in targetElement.srcObject &&
            typeof targetElement.srcObject.getTracks === "function" &&
            "addTrack" in targetElement.srcObject &&
            typeof targetElement.srcObject.addTrack === "function") {
            return targetElement.srcObject;
        }
        return null;
    }
    private saveRecordedVideo(ev: BlobEvent): void {
        if(ev.data.size <= 0) {           
            return;
        }
        const url = URL.createObjectURL(ev.data);
        const downloadTarget = document.getElementById("download_target") as HTMLAnchorElement;
        downloadTarget.download = "sample.webm";
        downloadTarget.href = url;
        downloadTarget.click();
    }
    private getMimeType(): MediaRecorderOptions {
        if(MediaRecorder.isTypeSupported("video/webm; codecs=vp9")) {
            return { mimeType: "video/webm; codecs=vp9" };
        }
        if(MediaRecorder.isTypeSupported("video/webm; codecs=vp8")) {
            return { mimeType: "video/webm; codecs=vp8" };
        }
        return { mimeType: "video/webm" };
    }
}

코덱

기본적으로 비디오는 WebM으로만 저장할 수 있습니다.
Firefox는 vp9를 처리할 수 없기 때문에 MIME 유형에 대해 vp8을 추가합니다.

MediaStream 합성

예를 들어 비디오에 사진, 오디오를 추가하고 싶습니다.

오디오

여러 비디오 트랙 또는 오디오 트랙을 하나의 MediaStream에 추가할 수 없습니다.
첫 번째 트랙만 취급합니다.

...
    public startRecording() {
        const localVideo = document.getElementById("local_video") as HTMLVideoElement;
        const localVideoStream = this.getVideoStream(this.localVideo);
        if(localVideoStream != null) {
            // these two rows are ignored,
            // because localVideoStream already has a video track and an audio track.
            localVideoStream.addTrack(someVideoStream);
            localVideoStream.addTrack(someAudioStream);

            this.recorder = new MediaRecorder(localVideoStream, this.getMimeType());
            this.recorder.ondataavailable = (ev) => this.saveRecordedVideo(ev);
            this.recorder.start();
        }
    }
...

그래서 Web Audio API를 사용하여 여러 오디오를 병합합니다.

video-recorder.ts

export class VideoRecorder {
    private recorder: MediaRecorder|null = null;
    private localVideo: HTMLVideoElement;
    private recording = false;
    private mixedAudioDestinationNode: MediaStreamAudioDestinationNode|null = null;

    public constructor() {
        this.localVideo = document.getElementById("local_video") as HTMLVideoElement;
        this.localVideo.onplay = () => this.init();
    }
    public startRecording() {
        this.recording = true;
        const videoTrack = this.getVideoStream(this.localVideo)?.getVideoTracks()[0]!;
        if(videoTrack == null) {
            return;
        }
        const newStream = new MediaStream();
        if(this.mixedAudioDestinationNode != null) {
            newStream.addTrack(this.mixedAudioDestinationNode.stream.getAudioTracks()[0]!);
        }
        newStream.addTrack(videoTrack);
        this.recorder = new MediaRecorder(newStream, this.getMimeType());
        this.recorder.start();        
        this.recorder.ondataavailable = (ev) => this.saveRecordedVideo(ev);
    }
    public stopRecording() {
        this.recording = false;
        this.recorder?.stop();
    }
...
    private init(): void {
        const localVideoStream = this.getVideoStream(this.localVideo);
        if(localVideoStream != null) {
            this.createMixedAudio(localVideoStream);
        }
    }
...
    private createMixedAudio(stream: MediaStream): void {
        const audioContext = new AudioContext();
        const audioSourceNode = audioContext.createMediaStreamSource(stream);
        const delay = new DelayNode(audioContext);
        delay.delayTime.value = 1;
        const splitter = audioContext.createChannelSplitter(2);
        audioSourceNode.connect(splitter);
        splitter.connect(delay, 1);
        const merger = audioContext.createChannelMerger(2);
        delay.connect(merger, 0, 1);
        splitter.connect(merger, 1, 0);
        this.mixedAudioDestinationNode = audioContext.createMediaStreamDestination();
        merger.connect(audioContext.destination);
        merger.connect(this.mixedAudioDestinationNode);
    }
...
}

영화

Canvas 요소에서 MediaStream을 만들 수 있습니다.
그리고 하나의 MediaStream에 여러 비디오 트랙을 추가할 수 없기 때문에
그래서 Canvas 요소에 이미지로 비디오를 그립니다.
이미지를 추가한 후 여기에서 MediaStream을 만듭니다.

video-recorder.ts

export class VideoRecorder {
    private recorder: MediaRecorder|null = null;
    private localVideo: HTMLVideoElement;
    private recording = false;
    private pictureCanvas: HTMLCanvasElement;
    private frameShown = false;

    private frameImage: HTMLImageElement|null = null;
    private mixedAudioDestinationNode: MediaStreamAudioDestinationNode|null = null;

    public constructor() {
        this.localVideo = document.getElementById("local_video") as HTMLVideoElement;
        this.localVideo.onplay = () => this.init();
        this.pictureCanvas = document.getElementById("picture_canvas") as HTMLCanvasElement;        
        this.pictureCanvas.style.position = "absolute";
    }
    public startRecording() {
        this.recording = true;
        const pictureStream = this.pictureCanvas.captureStream(60);
        const pictureTrack = pictureStream.getVideoTracks()[0];
        if(pictureTrack == null) {
            console.error("No picture video tracks");
            return;
        }
        const newStream = new MediaStream();
        if(this.mixedAudioDestinationNode != null) {
            newStream.addTrack(this.mixedAudioDestinationNode.stream.getAudioTracks()[0]!);
        }
        newStream.addTrack(pictureTrack);
        this.recorder = new MediaRecorder(newStream, this.getMimeType());
        this.recorder.start();
        this.updatePictureCanvas(this.pictureCanvas.getContext("2d") as CanvasRenderingContext2D);

        this.recorder.ondataavailable = (ev) => this.saveRecordedVideo(ev);
    }
    public stopRecording() {
        this.recording = false;
        this.recorder?.stop();
    }
    public updateCanvasSize(): void {
        this.pictureCanvas.width = this.localVideo.videoWidth;
        this.pictureCanvas.height = this.localVideo.videoHeight;
        const rect = this.localVideo.getBoundingClientRect();
        this.pictureCanvas.style.top = `${rect.top}px`;
        this.pictureCanvas.style.left = `${rect.left}px`;
        const ctx = this.pictureCanvas.getContext("2d") as CanvasRenderingContext2D;
        this.frameImage = new Image();
        this.frameImage.onload = () => this.drawFrameImage(ctx);
        this.frameImage.src = "../img/frame.png";
    }
    public switchFrame(): void {
        if(this.frameShown === true) {
            this.pictureCanvas.style.display = "none";
            this.frameShown = false;
        } else {
            this.pictureCanvas.style.display = "block";
            this.frameShown = true;
        }
    }    
    private init(): void {
        const localVideoStream = this.getVideoStream(this.localVideo);
        if(localVideoStream != null) {
            this.createMixedAudio(localVideoStream);
        }
    }
    private getVideoStream(targetElement: HTMLVideoElement): MediaStream|null {
        if(targetElement.srcObject != null &&
            "getTracks" in targetElement.srcObject &&
            typeof targetElement.srcObject.getTracks === "function" &&
            "addTrack" in targetElement.srcObject &&
            typeof targetElement.srcObject.addTrack === "function") {
            return targetElement.srcObject;
        }
        return null;
    }
    private createMixedAudio(stream: MediaStream): void {
        const audioContext = new AudioContext();
        const audioSourceNode = audioContext.createMediaStreamSource(stream);
        const delay = new DelayNode(audioContext);
        delay.delayTime.value = 1;
        const splitter = audioContext.createChannelSplitter(2);
        audioSourceNode.connect(splitter);
        splitter.connect(delay, 1);
        const merger = audioContext.createChannelMerger(2);
        delay.connect(merger, 0, 1);
        splitter.connect(merger, 1, 0);
        this.mixedAudioDestinationNode = audioContext.createMediaStreamDestination();
        merger.connect(audioContext.destination);
        merger.connect(this.mixedAudioDestinationNode);
    }
    private saveRecordedVideo(ev: BlobEvent): void {
        if(ev.data.size <= 0) {
            console.error("No video data");            
            return;
        }
        const url = URL.createObjectURL(ev.data);
        const downloadTarget = document.getElementById("download_target") as HTMLAnchorElement;
        downloadTarget.download = "sample.webm";
        downloadTarget.href = url;
        downloadTarget.click();
    }
    private getMimeType(): MediaRecorderOptions {
        if(MediaRecorder.isTypeSupported("video/webm; codecs=vp9")) {
            return { mimeType: "video/webm; codecs=vp9" };
        }
        if(MediaRecorder.isTypeSupported("video/webm; codecs=vp8")) {
            return { mimeType: "video/webm; codecs=vp8" };
        }
        return { mimeType: "video/webm" };
    }
    private updatePictureCanvas(ctx: CanvasRenderingContext2D) {
        if(this.recording === false) {
            return;
        }
        this.drawFrameImage(ctx);
        // To save as video, I have to redraw the images.
        setTimeout(() => this.updatePictureCanvas(ctx), 1000.0 / 60.0);
    }
    private drawFrameImage(ctx: CanvasRenderingContext2D): void {
        if(this.frameImage == null) {
            return;
        }
        ctx.drawImage(this.localVideo, 0, 0, this.localVideo.videoWidth, this.localVideo.videoHeight);
        ctx.drawImage(this.frameImage, 0, 0, this.localVideo.videoWidth, this.localVideo.videoHeight);
    }
}

자원

Media Capture and Streams

MediaStream Recording API - Web APIs|MDN

MediaRecorder - Web APIs|MDN

MediaStream - Web APIs|MDN

Using the MediaStream Recording API - Web APIs|MDN

Record almost everything in the browser with MediaRecorder - Mozilla Hacks - the Web developer blog

Reference

이 문제에 관하여([TypeScript] MediaRecorder로 MediaStream 저장), 우리는 이곳에서 더 많은 자료를 발견하고 링크를 클릭하여 보았다 https://dev.to/masanori_msl/typescript-save-mediastream-by-mediarecorder-13hh

텍스트를 자유롭게 공유하거나 복사할 수 있습니다.하지만 이 문서의 URL은 참조 URL로 남겨 두십시오.

우수한 개발자 콘텐츠 발견에 전념 (Collection and Share based on the CC Protocol.)

React에서 편집 가능한 테이블

대화 요소: 모달 및 팝업에 대한 새로운 접근 방식

좋은 웹페이지 즐겨찾기

개발자 우수 사이트 수집

개발자가 알아야 할 필수 사이트 100선 추천 우리는 당신을 위해 100개의 자주 사용하는 개발자 학습 사이트를 정리했습니다