Well, a RAM file is easy since it just contains an ASCII URL that redirects you to an RM file. An RealMedia (RM) file is a container format typically used for streaming audio and/or video with the Real Time Streaming Protocol (RTSP).
Now the A/V content is normally encoded and that's where a codec comes into play. There are probably a dozen different codecs that could be used in a RealMedia container, and most are likely protected intellectual property with hefty licensing fees.
Other than adding VoIP to an application using the speex codec and a quick and dirty implementation of Realtime Transport Protocol (RTP) many years ago, I don't have much experience in this realm. So, I don't know of any books/references to refer to you.
PS - I do remember discussing audio compression using convolution filters from my college signals and systems class though! :geek:
Edit: It may be easier to start with something very open with easily obtainable specifications like Ogg/Theora/Vorbis. In this case ogg is a container analogous to RM, theora is a video codec and vorbis is an audio codec. These are all open formats as opposed to proprietary RealNetworks IP.