Wikipedia:Reference desk/Archives/Computing/2013 March 6
Computing desk | ||
---|---|---|
< March 5 | << Feb | March | Apr >> | March 7 > |
Welcome to the Wikipedia Computing Reference Desk Archives |
---|
The page you are currently viewing is an archive page. While you can leave answers for any questions shown below, please ask new questions on one of the current reference desk pages. |
March 6
editVideo codecs for future storage
editPart 1
I'm getting several hours of super 8 film digitized. I want to keep it in a format that strikes a good balance between keeping the amount of storage space needed at a managable level, and preserving as much as possible of the information in the video. I've had tests done at several companies who specialize in this, and the company so far that produces the best digitizations, uses this hardware, and a codec that VideoLan describes thusly:
Stream: 0 Type: Video Codec: Packed YUV 4:2:2, U:Y:V:V (2vuy) Language: English Resolution: 1280x720 Display resolution: 1280x720 Frame rate: 24
edit: YUV 4:4:2 corrected to YUV 4:2:2
When I calculate the total number of pixels encoded, the file size is about 2.7 bytes per pixel.
- Do we have an article about this codec? I've read Chroma subsampling, but anything more specific?
- Is this a codec that can be expected to be readable a long time into the future?
- In this particular codec, are all frames represented equally faithful to the original, or are there key frames that are represented more faithfully, with diffs in between which use some kind of lossy compression with respect to the key frames? To clarify: The details in the encoding scheme, i.e. whether key frames and diffs are used or whether information is stored a sequence of entire images, does not matter to me. If the diffs are exact, it's fine.
- I'm happy with a file size of up to about 4 bytes per pixel. Are there codecs that are widely used, can be expected to be readable a long time in the future, and which can be manipulated with ffmpeg (Windows version from http://ffmpeg.zeranoe.com/builds/), that would be a better choice than the one described above?
Part 2
- For some reason, the company mentioned above only delivers the files in .MOV containers. Can I convert to .AVI losslessly with ffmpeg, and safely delete the original?
- Will ffmpeg be able to split longer files into shorter sequences losslessly?
Thanks, --NorwegianBlue talk 07:42, 6 March 2013 (UTC)
- 1a. I can't find a specific article but YUV has more information. 1b. Yes. 1c. It is uncompressed; you lose nothing except half the horizontal color resolution, which is normally not noticeable and which virtually all other codecs lose as well. Every frame is a key frame. 1d. 8-bit YUV 4:2:2 is two bytes per pixel. You could ask about 24-bit RGB, which would be 3 bytes per pixel, but it's possible that YUV 4:2:2 is the raw output format of their digitizer and converting to anything else would lose information. You could also ask about more than 8 bits per channel, but I think you would have more compatibility problems. 2a. I think ffmpeg -i input.mov -vcodec copy output.avi will work. 2b. Probably. At any rate there are tools that can. -- BenRG (talk) 18:47, 6 March 2013 (UTC)
- Thanks a lot, BenRG. I wonder what the extra 0.7 bytes per pixel are? I just discovered Mediainfo, which reports the format like this:
Extended content
|
---|
General Complete name : MyFile.mov Format : MPEG-4 Format profile : QuickTime Codec ID : qt File size : 8.41 GiB Duration : 3mn 23s Overall bit rate : 355 Mbps Encoded date : UTC 2013-01-24 16:33:18 Tagged date : UTC 2013-01-24 16:36:06 Writing library : Apple QuickTime ©TIM : 00:00:00:00 ©TSC : 24 ©TSZ : 1 Video ID : 1 Format : YUV Codec ID : 2vuy Duration : 3mn 23s Bit rate mode : Constant Bit rate : 354 Mbps Width : 1 280 pixels Height : 720 pixels Display aspect ratio : 16:9 Frame rate mode : Constant Frame rate : 24.000 fps Color space : YUV Chroma subsampling : 4:2:2 Compression mode : Lossless Bits/(Pixel*Frame) : 16.000 Stream size : 8.37 GiB (100%) Title : Apple aliasdatahåndterer Language : English Encoded date : UTC 2013-01-24 16:33:18 Tagged date : UTC 2013-01-24 16:36:06 Audio ID : 2 Format : PCM Format settings, Endianness : Little Format settings, Sign : Signed Codec ID : sowt Duration : 3mn 23s Bit rate mode : Constant Bit rate : 1 536 Kbps Channel(s) : 2 channels Channel positions : Front: L R Sampling rate : 48.0 KHz Bit depth : 16 bits Stream size : 37.2 MiB (0%) Title : Apple lydmediahåndterer / Apple aliasdatahåndterer Language : English Encoded date : UTC 2013-01-24 16:33:18 Tagged date : UTC 2013-01-24 16:36:06 Other ID : 3 Type : Time code Format : QuickTime TC Duration : 3mn 23s Time code of first frame : 00:00:00:00 Time code settings : Striped Title : Tidskodemediehåndterer / Apple aliasdatahåndterer Language : English Encoded date : UTC 2013-01-24 16:36:06 Tagged date : UTC 2013-01-24 16:36:06 |
- Could a silent audio stream consume so much space? I interpret the "Bits/(Pixel*Frame) : 16.000" output as a confirmation of your statement that the yuv compression should result in 2 bytes per pixel.
- The conversion to .AVI that you suggested worked, sort of. The file played fine in VideoLan, I compared a PNG snapshot with the corresponding frame in the original, and found the the files to be identical (binary diff). However, this particular content in an AVI container crashed MediaInfo (but not ffprobe). --NorwegianBlue talk 23:27, 6 March 2013 (UTC)
- You don't need a silent audio track whether it consumes a lot of space or not, include
-an
in yourffmpeg
command. ¦ Reisio (talk) 00:09, 7 March 2013 (UTC)- I know I don't need it, but I'm puzzled by the size of the files as delivered by the company who did the digitization. I don't want to mess with the original files until I'm 110% sure what I'm doing, but I'll try stripping it as you suggested on a copy of one of the files, to see if that's the explanation. Thanks! --NorwegianBlue talk 08:41, 7 March 2013 (UTC)
- 1280×720×24×16 = 353894400, which is consistent with the 354 Mbps rate quoted above, and 203s×354Mbps = 8.37 GB, so I think either you miscalculated or the file with the problem is a different one. Note that 1 Mbps = 106 bps, but 1 GB = 230 B. The audio track accounts for less than 1% of the file size, as shown by Mediainfo. It's possible they include it because some playback software wasn't designed to handle files without an audio track. You could at least downsample it to 8 KHz or something. -- BenRG (talk) 18:31, 7 March 2013 (UTC)
- I indeed miscalculated, and was about to come here and say, slightly embarrassed, "mystery solved". I used the frame rate of the source material (18 fps), which was part of the file names, and completely ignored the blindingly obvious fact that they had inserted extra frames, to adjust to 24 fps instead of 18 fps. But it's a good thing you beat me to it, since in doing so, you also spelled out how other relevant calculations, which I was unsure about, are done. Thanks a lot! --NorwegianBlue talk 20:23, 7 March 2013 (UTC)
- 1280×720×24×16 = 353894400, which is consistent with the 354 Mbps rate quoted above, and 203s×354Mbps = 8.37 GB, so I think either you miscalculated or the file with the problem is a different one. Note that 1 Mbps = 106 bps, but 1 GB = 230 B. The audio track accounts for less than 1% of the file size, as shown by Mediainfo. It's possible they include it because some playback software wasn't designed to handle files without an audio track. You could at least downsample it to 8 KHz or something. -- BenRG (talk) 18:31, 7 March 2013 (UTC)
- I know I don't need it, but I'm puzzled by the size of the files as delivered by the company who did the digitization. I don't want to mess with the original files until I'm 110% sure what I'm doing, but I'll try stripping it as you suggested on a copy of one of the files, to see if that's the explanation. Thanks! --NorwegianBlue talk 08:41, 7 March 2013 (UTC)
- You don't need a silent audio track whether it consumes a lot of space or not, include
How to simultaneously copy the text of all the alternatives in a Combo box (drop-down menu) on a web page?
edit
I use Windows7 and Firefox v19 and/or Microsoft Internet Explorer v9.
I need to copy the text of all the available options inn a Combo box ("drop-down list") and paste them into my local text editor.
For example I would like to simultaneously copy all the options:
Volvo
Saab
Opel
Audi
from the combo box in the Result field on the web page: http://www.w3schools.com/tags/tryit.asp?filename=tryhtml_select (Link found and visited: 2013-03-06)
(Of course: In most cases there are tens or sometimes maybe hundreds of options in the combo boxes that I need to copy).
How may I do this?
--46.15.136.227 (talk) 17:20, 6 March 2013 (UTC)
- (OP) Adding to my question at 23:21, 6 March 2013 (UTC) :
The example list (Volvo Saab Opel Audi), referred to above, only uses the HTML-tag: <select>…, but of course I also sometimes need to copy a combo box-list that is given in a script, like the one on the web page http://www.oft.gov.uk/ (visited: 2013-03-06) under: "I am looking for..." (with the options: "Anti-money laundering", "OFT Board", "Current Campaigns", …).
The solutions suggested, below, so far (Thank you all!), namely:
1. Screenshot and OCR.
2. Viewing the page source, searching, cuting and pasting (with or without the Firefox Firebug-Add-on).
they both do work, but have prooven to be far too time consuming and cumbersome in most cases.
I guess, what I wish for is the ability to just riht-click the combo box-arrow and choose: "Copy the whole selection list to the clipboard".
-- (OP) 46.15.136.227 (talk) 23:21, 6 March 2013 (UTC)
- (OP) Adding to my question at 23:21, 6 March 2013 (UTC) :
- If all you have access to is the finished app, try this: It's messy, but a screen grab (or multiples, if it doesn't fit on one page) followed to OCR could do it. If you have access to the source code, then better methods are available. StuRat (talk) 18:06, 6 March 2013 (UTC)
- You could just view the page source and copy the options and their surrounding HTML, which you would then of course have to edit out, either by some suitable searching-and-replacing in a text editor, or perhaps a spreadsheet if there are extra complications. AndrewWTaylor (talk) 19:41, 6 March 2013 (UTC)
- I do stuff like that pretty often. Usually (? always? depends what browser I guess) there's a "view source" via a menu or rightclick or something. When you've got the HTML source, if you're lucky, you can search for one of the combo box entries and find the place where they're all specified. If you're not lucky, they might be elsewhere in some CSS file or something that's transcluded into the HTML, in which case I'm out of my depth and somebody else might be able to continue the advice. Gzuckier (talk) 19:46, 6 March 2013 (UTC)
- If you use Firebug (a Firefox addon) it's a simple matter of rightclicking the select box, "Inspect with FireBug", copy-pasting the source code there, and getting rid off the tags using search-and-replace. The inspect will work whether or not the dropdown was in the original source code, or generated by JavaScript. 82.75.247.59 (talk) 20:01, 6 March 2013 (UTC)
- I have expanded my original question a little (see above, just below my first entry).
-- (OP) 46.15.136.227 (talk) 23:21, 6 March 2013 (UTC)
- On a page with a lot of OPTION tags like that, you can paste something like this into the address bar of your browser:
- javascript:var o=document.getElementsByTagName("option"); var r=""; for(var i=0;i<o.length;i++) { r+=o[i].innerHTML+"\n"; }; alert(r);
- In theory, this will then give you a pop-up box with all of the text between the OPTION tags. You should then (on a modern browser, like Chrome) be able to copy those directly and paste them into your text editor. Note that this won't work exactly on that w3schools page, because those OPTION tags are embedded in an IFRAME. --Mr.98 (talk) 23:55, 6 March 2013 (UTC)
- If you did need to go inside of an IFRAME like the one on that page — or if you just want to experiment with it — you could use this code instead:
- javascript:var o=document.getElementsByTagName("iframe")[1].contentWindow.document.getElementsByTagName("option"); var r=""; for(var i=0;i<o.length;i++) { r+=o[i].innerHTML+"\n"; }; alert(r);
- You would have to modify the number after the "iframe" (1 in this case) to refer to the right one (there are at least two IFRAMEs on that page, so in this case it is accessing the second one, because these are ordered starting from 0). --Mr.98 (talk) 00:03, 7 March 2013 (UTC)
- If you did need to go inside of an IFRAME like the one on that page — or if you just want to experiment with it — you could use this code instead:
openadserving.com malware
edithad this infesting me for a while now, but norton (as supplied by my isp and automatically updated) doesn't seem to do anything with it. I got it temporarily bypassed with the hosts file but would sure like to eradicate it from the system, but a quick googling comes up with a whole mess of different lists of files, registry keys, startup menu entries to delete, i.e. a different list from every source. So, does anyone know when any of the antivirus guys are going to get a handle on this, or why they can't? Gzuckier (talk) 21:28, 6 March 2013 (UTC)
- Have you tried Malwarebytes in safe mode? It's saved me from reinstalling Windows many times. I don't know if it will work in your case or not, but it's worth a shot. And from my experience/reading, Norton really isn't all that great - I would recommend Microsoft Security Essentials for real-time protection. Both AV's I've linked are free. ProtossPylon 00:00, 7 March 2013 (UTC)
- This is just my personal opinion, but I'm not sure I'd call MalwareBytes an antivirus program.--Rockfang (talk) 18:33, 7 March 2013 (UTC)
- What would you call it, then? That's the only description I can think of for it. ProtossPylon 23:10, 7 March 2013 (UTC)
- This is just my personal opinion, but I'm not sure I'd call MalwareBytes an antivirus program.--Rockfang (talk) 18:33, 7 March 2013 (UTC)
Unix question
editHi. I have to survey a network of computers; there are Windows boxes, Mac boxes, various flavours of linux, Ubuntu and Kubuntu. One thing I need to do is establish how many hard disks (not partitons, actual physical drives) are inside each box. This is easy peasy for the Windows and Mac ones, but is there a simple fast and dirty way of doing it via a terminal on the others? I have some (but admittedly not a lot) experience of using bash. Cheers pablo 23:40, 6 March 2013 (UTC)
fdisk -l | grep 'Disk /'
(as root / viasudo
),/sbin/blkid
, million other ways. :) ¦ Reisio (talk) 23:53, 6 March 2013 (UTC)- And the answer is
fdisk: illegal option -- l
... maybe I have less experience than I thought, or need a more foolproof solution! pablo 00:31, 7 March 2013 (UTC)
- And the answer is
- That's odd. /sbin/fdisk is part of util-linux, so it should work on any desktop or server Linux. Embedded Linux distributions may use the Busybox version instead, but even has the -l option too. -- Finlay McWalterჷTalk 00:42, 7 March 2013 (UTC)
- The "fdisk -l" one is questionable, as it identifies 24 out of my fileserver's 11 hard disks: the 11 physical drives, the three software RAID arrays built from those drives, and the 10 LVM logical volumes sliced out of the RAID arrays. --Carnildo (talk) 01:50, 7 March 2013 (UTC)
- You need admin access to run "fdisk -l" in most cases. You can try "sudo fdisk -l" if you're setup for it. Shadowjams (talk) 02:18, 7 March 2013 (UTC)
- I seem to recall some oddity about Unix that you had to mount a disk to see it. Is that still the case ? StuRat (talk) 02:33, 7 March 2013 (UTC)
- Not to see that it exists, or to see its partitions. Shadowjams (talk) 03:02, 7 March 2013 (UTC)
- No. In order to mount an HDD or partition, the OS has to know it exists. You can type cat /proc/partitions and see both mounted and unmounted hard drives and partitions. Hard disks in Linux are listed in /dev. For example, if I had two SATA HDDs in my computer, I would have /dev/sda and /dev/sdb. Each disk is divided into one or more partitions. So if sda were divided into two partitions, I would have /dev/sda1 and /dev/sda2. If they were IDE hard drives, they would be /dev/hda and /dev/hdb. However, as Carnildo noted, a RAID controller can be used to group multiple disks into one and lead Linux to believe that the array is a single disk. For example, the RAID may have three disks, each 500 GB in size. But they're in a RAID 0, so Linux thinks they're one 1.5 TB disk. You may think that's an academic exception, but it's actually common in server environments where Linux is often used. In such a case, you should use the command-line utility that came with your RAID card to determine how many disks are in the array. For example, if your RAID is made by 3ware, you use tw_cli. If it's made by LSI, you use megamgr. If you're not using a RAID (which most home users aren't) then you can just ignore that part about RAIDs and simply count the number of lettered disks (e.g., sda, sdb, etc.) to get the number of disks in your system. Reading the contents of /proc/partitions is my preferred way to do so.—Best Dog Ever (talk) 03:21, 7 March 2013 (UTC)
Thanks all. Reading /proc/partitions has sorted things out in most cases, but on a few machines /proc/partitions does not exist, don't know why. fdisk does not run with the -l parameter on these machines either. /dev contains many, many entries of the form /sd??, for example
sd10q
sd10r
sd10s
sd10t
sd129i
sd129j
sd129k
sd129l
sd129m
sd129n
sd129o
sd129p
sd130q
sd130r
sd130s
sd130t
sd130u
sd136a
sd136b
sd136c
sd136d
sd136e
sd136f
sd136g
sd136h
sd137i
sd137j
sd137k
sd137l
sd137m
sd137n
sd137o
sd137p
sd138q
sd138r
sd138s
sd138t
sd138u
sd144a
sd144b
sd144c
sd144d
sd144e
sd144f
sd144g
sd144h
sd145i
sd145j
sd145k
sd145l
sd145m
sd145n
sd145o
sd145p
sd146q
sd146r
sd146s
sd146t
sd146u
sd16a
sd16b
sd16c
sd16d
sd16e
sd16f
sd16g
sd16h
sd17i
sd17j
sd17k
sd17l
sd17m
sd17n
sd17o
sd17p
sd18q
sd18r
sd18s
sd18t
sd18u
sd1i
sd1j
sd1k
sd1l
sd1m
sd1n
sd1o
sd1p
sd2q
sd2r
sd2s
sd2t
sd2u
sd3a
sd3b
sd3c
sd3d
sd3e
sd3f
sd3g
sd3h
sd8a
sd8b
sd8c
sd8d
sd8e
sd8f
sd8g
sd8h
sd9i
sd9j
sd9k
sd9l
sd9m
sd9n
sd9o
sd9p
from this it seems as though these partitions all exist on 18 physical hard disks. Am I interpreting this correctly? It does't seem likely as the box doesn't look big enough! pablo 16:27, 7 March 2013 (UTC)
- If you type df does it list any physical devices in /dev under filesystem? What kind of box is it and what version of UNIX or Linux is it running? You can check by typing cat /proc/version or uname -a. I know BSD uses /dev/ada0, /dev/ada1, etc. Macs use /dev/disk0, /dev/disk1, etc.--Best Dog Ever (talk) 01:12, 8 March 2013 (UTC)
- Under Linux, "df" lists filesystems, not drives, and only those filesystems that are mounted. Further confounding the issue is that it lists loopback filesystems, which have a "/dev" entry but which correspond to a file. --Carnildo (talk) 02:38, 8 March 2013 (UTC)
- He can still use it to deduce which devices are being mounted as disks. In the left-column, it will list physical partitions currently mounted and he can then deduce what types of devices are being partitioned.—Best Dog Ever (talk) 03:24, 8 March 2013 (UTC)
- Aha! Mystery boxes are running SunOS. Entries in
dev/dsk
are of the formatc0t0d0s0
. I have d0, d1 and d2 = 3 disks which seems more feasible. Apologies if I made this more complicated than it is, I now have a solution for each of the different operating systems. Thanks! pablo 09:27, 8 March 2013 (UTC)
- Aha! Mystery boxes are running SunOS. Entries in
- He can still use it to deduce which devices are being mounted as disks. In the left-column, it will list physical partitions currently mounted and he can then deduce what types of devices are being partitioned.—Best Dog Ever (talk) 03:24, 8 March 2013 (UTC)
- Under Linux, "df" lists filesystems, not drives, and only those filesystems that are mounted. Further confounding the issue is that it lists loopback filesystems, which have a "/dev" entry but which correspond to a file. --Carnildo (talk) 02:38, 8 March 2013 (UTC)