New PC Build

[drvnbysound]

So the thought I had a few posts above regarding adding the HDDs to the Datastore didn't work. I can certainly add the additional physical HDDs to the Datastore, however, as you guys probably already know, the VMs can only be given access to Virtual HDDs (.vmdk's).

I will have to keep looking to see if there is another, possibly better, way to address this, but I am again thinking that I may just have to share the physical HDDs from the Host, and simply map the HDDs on the respective VMs. Unfortunately, this will certainly require some speed testing to see if this is really going to be feasible for I/O from the VMs. I was really hoping to be able to get away with using VMs and having (1) physical machine, and have a reduced hardware footprint, but it's slowly looking like it really may not work as expected and require more than 1 PC

 

[pete_c]

Very nice setup there drvnbysound!

Not sure if you have any old PC's around; but I noticed the licenses for XP Pro on a few of them the other day. Checked one out and it was good. The OEM llicenses now are also much cheaper today than just a year ago.

 

[rismoney]

I understand what you are saying. Historically on Windows this is not possible w/o third party software. There used to be a company HP acquired Polyserve that allowed a disk to be access simultaneously by multiple computers. This was an enterprise solution but no longer is sold to my knowledge. What you are asking for, is actually a complicated scenario because of potential disk lock management issues that arise if two computers access the same disk simulatenously. That said, on the Windows side there is a new technology only available in Windows Server 2008 R2 Enterprise in their cluster called Clustered Share Volumes. This won't really apply to you either.
So unfortunately you are left with SMB shares as your only option. On Linux more flexible mounting options may be available.

SanMelody and Openfiler may be potential other solutions, but I haven't looked at them.

FYI - I am a Systems Engineer and have close ties to Microsoft so feel free to ask any questions you may have and I will either try to address them directly or get an official answer from MS on it.

Regards,
Rich

 

[drvnbysound]

Very nice setup there drvnbysound!

Not sure if you have any old PC's around; but I noticed the licenses for XP Pro on a few of them the other day. Checked one out and it was good. The OEM llicenses now are also much cheaper today than just a year ago.

Thanks. Unfortunately, I really don't have any others around... I built my first PC back in about '04, which I didn't upgrade until a year ago when I built what was originally this machine. I've built quite a few for others since, but really none for myself. I do have one other PC that was given to me, so technically I do have access to (2) XP licenses, but that's about it. Honestly, knowing that I will have 8GB of RAM, and what seems to currently be plenty of HDD space, I really dont know if I want to even bother with using XP :unsure:

 

[drvnbysound]

I understand what you are saying. Historically on Windows this is not possible w/o third party software. There used to be a company HP acquired Polyserve that allowed a disk to be access simultaneously by multiple computers. This was an enterprise solution but no longer is sold to my knowledge. What you are asking for, is actually a complicated scenario because of potential disk lock management issues that arise if two computers access the same disk simulatenously. That said, on the Windows side there is a new technology only available in Windows Server 2008 R2 Enterprise in their cluster called Clustered Share Volumes. This won't really apply to you either.
So unfortunately you are left with SMB shares as your only option. On Linux more flexible mounting options may be available.

SanMelody and Openfiler may be potential other solutions, but I haven't looked at them.

FYI - I am a Systems Engineer and have close ties to Microsoft so feel free to ask any questions you may have and I will either try to address them directly or get an official answer from MS on it.

Regards,
Rich

Thanks for the response, not only the last one, but also for following along and providing input throughout this thread. While I guess, technically, what I mentioned before would seem that that HDD would be able to be addressed by 2 PC's, I really don't care that they are. I was simply trying to find a solution to allow the VM direct access to the RAW HDD without the need for any special file type (e.g. Virtual Disk filetype .vmdk), so that if I ever decided that I no longer wanted the data hosted on a VM (not sure what scenario would change this) it would be easy to give access directly, without complication (such as having to re-copy files from a VM disk, to a shared HDD back on the host).

I am going to do some, not so intricate, testing tonight to see how quickly the VM can copy large amounts of data (e.g. 3+ ripped DVDs) from the local VM disk over to the shared/mapped HDD. If this is comparable to what the results are on the host OS, between physical HDDs, I think it should be sufficient for media server usage. Obviously, I would end up doing some 'real-world' testing with DVR functionality and playback of movies later, before actually implementing it as a home solution for my wife and I to start relying on - I would hate to get scolded for missing one of her recordings

Note, that I never plan to use any of the VMs or any other PC as a stand-alone extender for any media; I would use dedicated extenders such as the HD-300's, Western Digital's HD TV, or other, so I believe it really is the read/write speed to the HDD that will likely be my biggest concern (for now anyway) with regard to hosting this on a VM.

 

[drvnbysound]

Well, I dont want to COMPLETELY rule anything out yet, but initial findings look like VMs just arent going to work for what I am trying to do. I just did some testing copying 4 movies (~20.7GB) to a couple of different locations and found the following:

Using the Host OS, copying the movies from the local 2TB drive, over to the other local 2TB drive, took approximately 4 minutes. Doing some rudimentary calculations, if I did them correctly, that's approximately 88MB/sec

After copying the movies to the secondary 2TB drive, I selected to share this folder on the network.

Using the VM, I located the shared drive, and started to copy the movies from the Shared 'network' drive to the local C:/ of the VM... I cancelled the transfer after 10 minutes, and the status window was showing that it was ALMOST 1/2 done, and approximate time remaining was ~30 minutes. Sure, we all know that these estimations are just that, but after taking 2x the amount of time, and seemingly not being 1/2 done, didn't seem adequate. Note that the current Host OS configuration has 4GB of RAM installed, and I had only allocated 1GB of that to the VM... knowing that, the RAM gadget on the Host OS was >90% for the 10 minute duration.

Unless I can come up with some seemingly drastic performance upgrades, I don't see how using a VM to function as a media server is going to work.

For 'smiles and grins' I am going to try to add the 2TB drive to the VM Datastore, and see if there is any benefit to using them over shared drives.

 

[pete_c]

I keep my movies, music and pictures on separate NAS boxes; mostly running Linux/FreeBSD/FreeNas and W2003 server (one is W2003 embedded).

Not to sound like a server hugger; but I also keep TVersity and PlayOn TV on a separate box.

For media UPNP players I utilize Patriot Home Box Office boxes ($50 on sale) (similar to WD except for the 1 TB drives inside of them) (each have 1 TB drives - overkill a bit) tweaked a bit with remote web access, remote control and chrooted linux running on them. The network is mostly at Gb; the PBO's though run at 100Mb. They stream HD/DiVx fine from the NAS boxes. Mostly lately just utilizing the VM's for testing OS and new setups. There is one by each television in the house.

Initially I was using TVersity mostly to serve up audio; and recently went to video. With this earlier set up I couldn't do video very well; so went to a core duo with 4Gb of RAM on a small footprint mITX "green" PC. Its doing well.

 

[rismoney]

Well, I dont want to COMPLETELY rule anything out yet, but initial findings look like VMs just arent going to work for what I am trying to do. I just did some testing copying 4 movies (~20.7GB) to a couple of different locations and found the following:

Using the Host OS, copying the movies from the local 2TB drive, over to the other local 2TB drive, took approximately 4 minutes. Doing some rudimentary calculations, if I did them correctly, that's approximately 88MB/sec

After copying the movies to the secondary 2TB drive, I selected to share this folder on the network.

Using the VM, I located the shared drive, and started to copy the movies from the Shared 'network' drive to the local C:/ of the VM... I cancelled the transfer after 10 minutes, and the status window was showing that it was ALMOST 1/2 done, and approximate time remaining was ~30 minutes. Sure, we all know that these estimations are just that, but after taking 2x the amount of time, and seemingly not being 1/2 done, didn't seem adequate. Note that the current Host OS configuration has 4GB of RAM installed, and I had only allocated 1GB of that to the VM... knowing that, the RAM gadget on the Host OS was >90% for the 10 minute duration.

Unless I can come up with some seemingly drastic performance upgrades, I don't see how using a VM to function as a media server is going to work.

For 'smiles and grins' I am going to try to add the 2TB drive to the VM Datastore, and see if there is any benefit to using them over shared drives.

The core issue here is that 1GB network is only 125MB/sec assuming theoretical perfect conditions. Certainly not from a home router/switch-expect small fractions... In VMWare the switching is done in software. Couple that with if your source destination are both on the same host, it is still passing it up and down the network stacks. This is why I reccomended either 2 NICs and or 10G nic in a post higher in the thread. Virtualization is all about sharing resources, and you really need dedicated resources if you want to get the performance you are expecting.

A couple of other things that could help change things regarding tuning. First is if you plan to move big files, Jumbo Frames might help since packets won't be fragmented to 1492 and can instead go up to 9000bytes or so. Also on Win7 there are issues with autotuning particularly when going to legacy (XP) boxes. XP is limited to 65535 byte window sizes. Looking there might yield some performance improvments. Autotuning can be turned on/off with netsh on Win7. Jumbo is on the network adapter. Also ensure you are on latest Service Packs, as many include TCP/IP.sys updates.

Rich

 

[drvnbysound]

The core issue here is that 1GB network is only 125MB/sec assuming theoretical perfect conditions. Certainly not from a home router/switch-expect small fractions... In VMWare the switching is done in software. Couple that with if your source destination are both on the same host, it is still passing it up and down the network stacks. This is why I reccomended either 2 NICs and or 10G nic in a post higher in the thread. Virtualization is all about sharing resources, and you really need dedicated resources if you want to get the performance you are expecting.

A couple of other things that could help change things regarding tuning. First is if you plan to move big files, Jumbo Frames might help since packets won't be fragmented to 1492 and can instead go up to 9000bytes or so. Also on Win7 there are issues with autotuning particularly when going to legacy (XP) boxes. XP is limited to 65535 byte window sizes. Looking there might yield some performance improvments. Autotuning can be turned on/off with netsh on Win7. Jumbo is on the network adapter. Also ensure you are on latest Service Packs, as many include TCP/IP.sys updates.

Rich

Rich,

Thanks again for the response... I honestly didn't even think through the entire 'system' enough to realize that the transfer of the files from the VM to the shared folder on the Host OS, would be exactly that, and utilizing the VM (software) network to do so. I will add another NIC tonight and try it again.

I really dont plan to be moving large files, such as this often. So I really need to get my HDHR attached, and maybe either the old trial version of Sage or just use Media Center to see how well the VM performs when actually performing DVR functions.

 

[drvnbysound]

I was able to locate a few Gigabit PCI NIC cards that I had. However, I found that my motherbaord only has (2) PCI slots, so I was only able to add 2 of them. Now that these are installed I have 3 Gigabit ports (including the one on the motherboard).

I have not been successful in figuring out how to allocate the additional NIC card(s) to a VM. Via Google searches, I have come across some OLD threads that state how to do it when using VMware Server 1.0, but nothing for 2.0 - and the menus discussed are no longer available in 2.0. When using VMware Server 2.0 it seems that you have to manage the networks through a separately installed application, 'Manage Virtual Networks'. Unfortunately, it's not as simple as locating the newly installed NIC cards and adding them to an inventory list for the VMs to utilize.

If anyone has any insight on how to add these to the VM inventory and/or allocate them to specific VMs, please advise. Until then, I will continue to beat my head against my keyboard

EDIT: I think I have just made SOME progress, as I found that I have to run the application as the Administrator, allowing additional options to be selected.

EDIT 2: I was able to get the NIC adapters added, and I have one of them setup and working with VM#1. I have confirmed that it is utilizing the NIC card by disconnecting the connection and seeing network loss (webpages not loading). I have also confirmed that the VM is not effected when the other NIC ports are disconnected. I do need to go through some more network setup as current configuration only allows directional PINGs to complete; I can ping the Host from the VM, but cannot ping the VM from the Host. As I will need to provide access to the VM from external devices (media extenders) and provide access to the VM from external tuners.

Also, does it make sense that the VM shows that the NIC is an Intel Pro/1000 MT Network Card, even though the Physical NIC installed on the Host is a D-Link DGE-530T? I assume the Intel is just the standard NIC that VMware somehow provides to the guest OS, regardless of which physical NIC is actually used (??)

 

[pete_c]

When I test I typically utilize my home office desktop with only one NIC. I have VM Workstation installed. At "work" I always used a multiple NIC server; but requested the specfic build and never personally configured it.

Looks like you are having an issue with using NAT / bridging the NIC interface.

I do see the NIC in the VM defined as a VM virtual NIC defined to the true AMD NIC that exists on the box here and I can ping from/to the interfaces when running a MS VM or a Linux VM.

In the device list on VM for Workstation I see a list of NICs with a VM0, VM1, VM2 definition. When I bring up the VM I see it using VM0; but if I look at the actual VM OS then I see it as the hardware as defined interface (AMD).

 

[rismoney]

Also, does it make sense that the VM shows that the NIC is an Intel Pro/1000 MT Network Card, even though the Physical NIC installed on the Host is a D-Link DGE-530T? I assume the Intel is just the standard NIC that VMware somehow provides to the guest OS, regardless of which physical NIC is actually used (??)

I believe this is normal. The hypervisor [VMWare] doesn't present physical devices to the guests. Instead it presents virtual devices and these often times use the most generic of drivers. I will check my systems at work tomorrow, but I have never installed NIC drivers on guests. When I push out the VMWare Tools to the guests it should take care of everything driver related and optimize it.

Note, you can also look into bonding for load balancing, fault tolerance, both, etc. This can combine 2 1GB nics into a 2GB Nic. You have a ton of flexibility and can test different performance scenarios. Google around.

 

[fwd03]

What rismoney said. You definitely have a lot of testing to do, some CCTV software packages don't like being hosted in a VM, and while SageTV will run in a VM, if you plan on using any tuners, you probably want to avoid doing this (plus disk speed won't be that great either unless you invest in a massive storage setup).

Dan is correct. I had one web server would not work on Microsoft VM, but works fine on a Windows server running on its own hardware.