This is a specification of Modular library for platform-dependent audio video and desktop capture GSoC idea.

Should be written using platform native APIs, i.e. no GStreamer, OpenCV, QtMultimedia, etc.

Windows:

• Windows Media Foundation (Windows Vista+), C++ API
• DirectShow (Windows XP+), C++ API

OS X:

• AV Foundation (OS X 10.7+), Obj-C API

• Get a list of all camera devices, with their respected camera ids and UI-friendly device names.
• Specify which camera to use, based on the camera id from the camera list.
• Use multiple cameras at once.
• Capture a single frame (a still image).
• Capture a continuous stream (a video).
• All capturing should be done in YUV and RGB.
• Get list of supported camera resolutions.
• Set resolution of capturing images.
• Get notified when a camera is connected to/disconnected from the computer.
• Change image brightness, contrast, saturation.

Should be written using platform native APIs (what about OpenAL?).

• Get a list of all audio recording devices, with their respected device ids and UI-friendly device names.
• Specify which audio recording device to use, based on the device id from the device list.
• Capture audio in 48khz 16bit signed PCM.
• Get a list of all audio playback devices, with their respected device ids and UI-friendly device names.
• Specify which audio playback device to use, based on the device id from the device list.
• Audio playback.
• Audio mixing.
• Get notified when an audio device is connected to/disconnected from the computer.

Should be written using platform native APIs. On Linux it should support X11, Wayland and Mir.

• Get a list of all monitors.
• Specify which monitor to capture, based on the list.
• Capture a single frame.
• Capture a continuous stream. Some(?) APIs provide this, which is different from manually capturing frame by frame.
• All capturing should be done in YUV and RGB.
• Get monitor's properties, such as resolution.
• Get notified when a monitor is added/removed.