The Covid-19 pandemic got me scrambling to come up with some web based solutions to support local Summer art festivals. In April I started digging into WebRTC as a potential in-browser low-barrier-to entry Real Time video solution. After some successful proof of concepts, I realized browser support and all the bits necessary to bring my idea to life were going to require me to learn a lot more than I knew and dedicate a lot more time than I had. I ended up hacking together a wrapper on top of the Jitsi platform and it worked well enough to help out some artists. However, I am now back on a quest to explore WebRTC more fully.
In this post, I dive into some basics and ultimately demonstrate how to generate a list of available hardware devices for a User. I put together a demo on my site which is also a handy tool to help diagnose browser access to devices.
High Level API
First we need to identify some objects and types we'll be working with. Note: That this is all available natively in the browser but some of the Types are from TypeScript.
navigator
(docs) A global of typeNavigator
containing info about the application running the script.navigator.mediaDevices
(docs) - a property of typeMediaDevices
which provides access to connected media devices like cameras, microphones, headsets, etc, as well as screen sharing.MediaConstraints
(docs) An object containing information about what devices we want to access and requirements for these devices. This object can define, for example, that we prefer to access the front facing camera or a known audio input.navigator.mediaDevices.getUserMedia()
(docs) A function that returns aPromise<MediaStream>
containing information about the devices based onMediaConstraints
argument. When executed, this will prompt the user to Allow or Block access to the particular kind of devices requested (eg. camera or microphone). Note: If the user has previously Blocked access to a particular device kind, this may skip the prompt and eject the promise immediately .navigator.mediaDevices.enumerateDevices()
(docs) A function that returns aPromise<MediaDeviceInfo[]>
. In Chrome, if the user has not yet Allowed access for the device kind, this may still resolve, but not contain complete information (such as missing device label). In Firefox, this will resolve with complete info even if the user Blocked access.MediaDeviceKind
(docs) An enum of values forMediaDeviceInfo.kind
. Allowed values areaudioinput
,audiooutput
, andvideoinput
A Simple TypeScript Example
Below is a basic example of requesting access to user's available microphones and cameras and logging out the available devices. Note: As a security measure, the user must interact with the page prior to calling getUserMedia
or else it will error. As such, this is wrapped in an init() function.
// Enum of available MediaDeviceInfo.kind values
enum MediaDeviceKind {
CAMERA = 'videoinput',
MICROPHONE = 'audioinput',
SPEAKER = 'audiooutput',
}
// Helper Method to log out devices
const getAvailableDevices = () => {
// NOTE: Even if access is denied, there may be records depending on browser
let enumeratorPromise = navigator.mediaDevices.enumerateDevices();
enumeratorPromise.then((devices: MediaDeviceInfo[]) => {
console.log(devices);
});
};
const init = () => {
// Get User Media status for Audio
navigator.mediaDevices
.getUserMedia({ audio: true, video: false }) // MediaConstraints
.then(handleSuccess(MediaDeviceKind.MICROPHONE))
.catch(handleError(MediaDeviceKind.MICROPHONE))
.finally(() => {
getAvailableDevices();
});
// Get User Media status for Video
navigator.mediaDevices
.getUserMedia({ audio: false, video: true }) // MediaConstraints
.then(handleSuccess(MediaDeviceKind.CAMERA))
.catch(handleError(MediaDeviceKind.CAMERA))
.finally(() => {
getAvailableDevices();
});
// Finally Update state with all media devices.
getAvailableDevices();
};
// getUserMedia Promise callbacks
function handleSuccess(deviceType: MediaDeviceKind) {
return (stream: MediaStream) => {
console.log([deviceType, stream]);
// For this Demo, we don't actually do anything with the stream and want to release the device
// Note: This will cause the camera to momentarily blink
stream.getTracks().forEach(function (track) {
track.stop(); // e.g. turn camera off, etc
});
};
}
function handleError(deviceType: MediaDeviceKind) {
return (error: DOMException) => {
// An error occurred getting one or more media based on constraint.
let errMsg =
'Encountered Error while attempting to getUserMedia for ' + deviceType + ' ' + error.message + ' ' + error.name;
console.error(errMsg)
};
}
Notes:
- Even though
MediaDeviceKind
is an existing TypeScript type, I redefined as an enum so it can be used as a value in the code. - I called
getUserMedia
separately for the camera and for the microphone. This was partially to more easily keep track of which device succeeded or failed, but also demonstrates that you can call getUserMedia only for what you need to access when you need to access it. - Similarly, I wrapped the
getUserMedia
handlers to return the actual callback function so it was easier to determine what device kind succeeded or failed.
An Example
I put together a code lab to serve as a proof of concept as well as an easy way to debug different devices across browsers.
Once clicking "Start", to initialize the UI, you are prompted for access to the camera and mic.
If access is granted to both the mic and camera, the devices will be displayed:
If you reject either the camera or the mic, the respective getUserMedia
promise rejects:
If I plug in my external USB webcam and refresh, the device will appear in the list as well:
Finally, if I plug in my Focusrite Scarlett audio interface, I see it as a microphone and speaker (see note below about Chrome and speakers):
Awesome!
Some Cross Browser Notes
There are some minor differences between browsers worth noting:
- Safari and Firefox do not appear to return any devices of kind
audiooutput
. Only Chrome seems to keep track of that. Support for targeting output devices seems spotty. - If you deny access to either the mic or the camera, but not both Firefox and Safari will show you the device labels. Whereas, Chrome only shows the device labels if you allowed access to the specific kind of device. This feels like a security flaw potentially.
- I have not tested any of this in Internet Explorer nor Edge. Feel free to post in the comments if my code lab tool works for you in these browsers.
Conclusion
While functionality varies between browsers, it is fairly easy to get a list of hardware devices for a user as long as they fall under the category of "microphone" or "camera". This was a pretty easy starting point for digging into WebRTC. In the next post in this series, I will be leveraging these different devices to locally record audio and video and do some basic manipulations.
p.s. I'm really excited about this type of work. If you have a project working with WebRTC, especially if it is helping make the world better right now, hit me up.
Image Credit: Photo by pascal claivaz from Pexels