Amazon Echo to HA Controllers

[ChrisCicc]

Microsoft has never made the "best" SAPI TTS voices. Just really good free ones. For years the best was Ivona, and they were priced accordingly. Many of our customers use the Brian voice to imitate Jarvis, it's great.
 
How did Amazon get so good so quickly? They bought Ivona.
 
In other words, you can install the same quality voice the Echo has, on Windows. 

 

[pete_c]

Remembing playing with Dragon in the 1990's. 
 
Thinking too it was offered for IBM's OS2 Warp at the time.  (Also Windows 95 and Amiga stuff?).
 
Interesting stuff about Dragon .... note this is a bit off the OP.....
 
Dr. James Baker laid out the description of a speech understanding system called DRAGON in 1975. Then in 1982 he and Dr. Janet Baker, his wife, founded Dragon Systems to release products centered around their voice recognition prototype; he was President of the company and she was CEO. DragonDictate was first released for DOS, and utilized hidden Markov models, a probabilistic method for temporal pattern recognition. At the time, the hardware was insufficiently powerful to address the problem of word segmentation, and DragonDictate was unable to determine the boundaries of words during continuous speech input. Users were forced to pronounce one word at a time, each clearly separated by a small pause. DragonDictate was based on a trigram model, and is known as a discrete utterance speech recognition engine.

Dragon Systems released NaturallySpeaking 1.0 as their first continuous dictation product in 1997. The company was then purchased in June 2000 by Lernout & Hauspie, a Belgium-based corporation that had been involved in financial scandals as reported by the New York Times. Following the all-share deal advised by Goldman Sachs, Lernout & Hauspie declared bankruptcy in November 2000. The Bakers had received hundreds of millions in stock, but were only able to sell a few million dollars worth before the stock lost all its value as a result of the accounting fraud. The Bakers sued Goldman Sachs for negligence, intentional misrepresentation and breach of fiduciary duty, which eventually (January, 2013) led to a 23-day trial in Boston. The jury ruled in favor of Goldman Sachs and cleared them of all charges.Following the bankruptcy of Lernout & Hauspie, the rights to the Dragon product line were acquired by ScanSoft of Burlington, Massachusetts. In 2005, ScanSoft launched a de facto acquisition of Nuance Communications, and rebranded itself as Nuance.

 

[deandob]

I also used Ivona TTS voices for HA responses, agree they are the most natural sounding.
 
Seems like the old SAPI approach (local recognition / TTS) is now being overtaken by the cloud services like Amazon, Cortana (Microsoft), Google and Apple Siri. I have done some experiments with Google voice and recently with Cortana in Windows 10, and the recognition has been very accurate and also surprisingly fast as I expected some latency between speaking and recognition and the streaming TTS response - seems the speech is streamed to the Cortana cloud with very low latency and you can see the words being recognised on the bottom of the Cortana toolbar almost instantaneously, quite impressive, and certainly fast enough for a responsive HA system with VR and TTS. I expect Amazon is doing similar with Echo, I'll find out when mine arrives and I hack into it to see how it works. My plans are to use either Raspberry Pi2 with Win 10 IoT and Cortana (once released) for local room VR / TTS or perhaps the Echo.
 
I see others are using Kinect for VR, which likely uses the same Cortana cloud, however having done a lot of playing (hacking) with the Kinect for HA I don't like it's form factor for in room automation, so I'm wondering how others are integrating it into their living spaces and how you handle the USB to the computer + Kinect power needed?? The idea behind Kinect for VR especially with its mic array is good, I just can't see the form factor being practical.

 

[pete_c]

I see others are using Kinect for VR, which likely uses the same Cortana cloud, however having done a lot of playing (hacking) with the Kinect for HA I don't like it's form factor for in room automation, so I'm wondering how others are integrating it into their living spaces and how you handle the USB to the computer + Kinect power needed?? The idea behind Kinect for VR especially with its mic array is good, I just can't see the form factor being practical.

 
Here purchased 3 Kinects, multitouch openframe thin monitors and one Intel Baytrail PipoX7 for testing last year.  The low powered Intel Baytrail will be velcroed to the back of the thin monitor and has multiple USB ports, LAN, bluetooth and wireless built in and came with Wintel 8.1 lite.  These will be connected to the Kinects.  Really the largest piece is the Kinect and the LCD monitor.
 
The look to the Kinects (for me) started here looking at Chris's CastleOS automation software.
 
Always got in to hardware. 
 
In the 1980's had a BBS in Chicago which I tapped in to a bunch of pay phone lines, used multiple Ventel modems and built a little hardware switcher for multple phone lines.  This was hobby related and I was a bit into phone stuff at the time. I also used a CPM computer for bean counting (work related stuff) at the time. 
 
Here newest version of Homeseer (started to use it in 1998) lets you run plugins remotely.  Works fine for me in Linux as I am already doing this with other plugins.  The Kinect plugin will be running on the Intel Pipo X7 and be talking to the mothership running Ubuntu 14.04 64 bit Linux.  Today same box is also running a Windows server VB for just the Microsoft SAPI voices and works fine.  The 20 plus Wintel embedded touchscreens connecting to the mother ship all have MS SAPI on them.  These are unique devices which have been modded.  The tabletop tablets are Openpeak tablets which were used for telco or utility kioks. 
 

 

 
They have an Intel Atom based motherboard with Zigbee, DECT, WLAN, Bluetooth and Gb NIC.  The original OS was an EFI boot to 1-2 Gb flash MMC.  I redid the EFI boot to a Seabios '86 boot and originally soldered on a PATA port connecting an SSD drive inside (which fit fine).  John Skully had left Apple and was doing this stuff for Openpeak to stir up some competition.  IE: Ayaya and Cisco rebranded these devices for hotel room use.  Utility companies used these for energy monitoring.  IE: they also have cellular SIM slot traces / ciruits for use.  Little marvel Intel tablet top touch screens.
I stream live TV / CCTV on these from my MyTV box just fine.

 

[deandob]

Pete, some good hacking there! So basically you have smart consoles distributed around the house and you use a homeseer plugin that remotes the Kinect from a central server to the consoles?
 
Physically how do you integrate the Kinects into your lounge room, bedroom etc? My wife wouldn't put up with a device like a Kinect in a prominent location.
 
What is the functional difference between your 20 Intel tablets you have modified and the 3 monitors you use with the Kinect (apart from obviously the Kinect)?
 
Your intel tablets remind me of the my first HA implementation, I purchased 10 years back 16 3Com Audreys at a liquidation sale when they were discontinued and hacked them as clients for a Windows Server running v1 of my HA software. The 3Coms ran QNX and I wrote a C client to talk sockets back to the Server and used the Audrey as a smart client - in fact the trigger word for the house VR is still 'Audrey' (actually the phrase 'Audrey Listen'). TTS Ivona SAPI voices ran on the Windows Server and fed back to the client via a balanced audio line. The Audrey was cute & quite advanced for its time but underpowered and a crummy resistive touchscreen & passive LCD screen. I still have them boxed up in a cupboard (if anyone is interested in buying them) as I changed to PIC microchips for room controllers with voice control & text based LCD screens instead for v2 of my HA system, which are now being replaced with dedicated in-wall 5" capacitive touchscreens the size of a light switch with the v3 HA software I have posted about in another thread.
https://en.wikipedia.org/wiki/3Com_Audrey

 

[ChrisCicc]

Physically how do you integrate the Kinects into your lounge room, bedroom etc? My wife wouldn't put up with a device like a Kinect in a prominent location.

 
Don't put it in a prominent location, it blends in far better than you might think at first glance. You can also hide it on the top of furniture, wall mount it, etc... 

 

[pete_c]

Pete, some good hacking there! So basically you have smart consoles distributed around the house and you use a homeseer plugin that remotes the Kinect from a central server to the consoles?

yes...thank-you ...always liked to tinker...
 

Physically how do you integrate the Kinects into your lounge room, bedroom etc? My wife wouldn't put up with a device like a Kinect in a prominent location.

 
I have not done this yet.
 

What is the functional difference between your 20 Intel tablets you have modified and the 3 monitors you use with the Kinect (apart from obviously the Kinect)?

 
The monitors are larger touchscreens.  No functional differences.  Just switching 4:3 in wall monitors to 16:9 monitors.
 

Your intel tablets remind me of the my first HA implementation, I purchased 10 years back 16 3Com Audreys at a liquidation sale when they were discontinued and hacked them as clients for a Windows Server running v1 of my HA software. The 3Coms ran QNX and I wrote a C client to talk sockets back to the Server and used the Audrey as a smart client - in fact the trigger word for the house VR is still 'Audrey' (actually the phrase 'Audrey Listen').

 
Many Homeseer users used Audreys in the early 2000's.  Here played with a tablet device called an ePOD that had Windows CE and later a tablet that ran Linux or Windows called a Sonic Pro Progear.. (used a Transmeta CPU) (purchased a few of these for Homeseer).  Thinking I saved one with a docking cradle.  It also has a resistive screen.  Here also used a tiny touch screen computer called a Panasonic CF-01 (years before the Transmeta?) and it was more IBM than Panasonic at the time  (well purchased a few of these and other early tablets that I used for Homeseer).
 
Used to frequent a site called linuxhacker dot org way long time ago where folks there would modify all sorts of devices.
 
Yup; here still running multiple MS SAPI TTS fonts on the Ubuntu 14.04 64 bit Homeseer mothership.  That said all of the Wintel touch clients also run MS SAPI TTS voice fonts.  I have two older Homeseer 2 boxes still running with SAPI and just use a cheapo audio mixer for all of the TTS chit chat....might do that with the Amazon Echo (just put the audio in to the audio mixer).
 
Here also use HAI Omnitouch 5.7/5.7e's plugged in to the OmniPro II.  Wife likes the Omnitouch touchscreens more than mine as I put too much stuff on mine which I shouldn't.  Wife is very patient with me.
 

....which are now being replaced with dedicated in-wall 5" capacitive touchscreens the size of a light switch with the v3 HA software I have posted about in another thread.

 
Very nice.
 
What are you powering these with OS / CPU wise?

 

[deandob]

I'm having some trouble sourcing a decent 5" screen with capacitive touch and able to be mounted portrait (like a light switch) either IPS LCD or a LCD with vertical polarisation. Will either be beaglebone black (running Linux / node.js) or Raspberry Pi 2 (running Windows 10 IoT and the new MS universal app model). Still prototyping both the hardware & software at the moment so am a couple of months away from anything finished. Unit will also have a class D amp for in wall speakers + a few other peripherals like light sensor, IR receiver, mic preamp tuned for VR (filter, level gate & AGC) & PCBs made specifically for the design and form factor.

 

[pete_c]

I'm having some trouble sourcing a decent 5" screen with capacitive touch and able to be mounted portrait (like a light switch) either IPS LCD or a LCD with vertical polarisation.

 
It's been a couple of years now that I purchased 4:3 15" / 17" capacitance touch screens with 3M touch.  These were all openframe and labeled as testing prototypes.  Very thin.  They also sold 5" / 7" same style openframe capacitance prototypes.  The hardware connectivity piece on these is small allowing you to flush mount the device and use a small square hole in the wall for connectivity. 
 
This person did an in wall thing with a 2.5" touch monitor.  I am now seeing small capacitance screens for the RPi2.  Here favorite OmniTouch screen was the 3.5" one over the 5.7" one.  (resistive though)
 

 
Here still utilize commercial style Liliput 9" resistive screen's.  They are made for point of sale machines and very modular.
 
I don't really know if the smaller one's got sold.  These monitors though had VGA / DVI inputs / USB touch ports on them. 
 
A while ago also tested a Mimo USB touch screen monitor connecting it to a Pogo Plug (Seagate Dockstar) for use with Homeseer.  These worked fine.  Here installed a base Ubuntu and RDP'd over to a Wintel RDP server.  They are resistive type of monitors and they do make capacitance style monitors today.  The issue at the time I had related to the power draw of the device.  There are are few Homeseer users still using these for touchscreen consoles.  Thinking they still make these monitors.  Note that these are 7" monitors.
 

 
 
Will look for the source vendor to see if they are still around.
 
BTW you can also purchase an old Joggler and just utilize the capacitance screen from it.  It is larger than 5" but you can find these now for around $30.  The capacitance touch controller / backlight stuff is built in to the back of the thin capacitance screen except that the cabling to the motherboard is not documented. It is standard plus and includes powering up the controller and backlight wires power stuff.  Well that and the schematics are not documented anywhere that I could find.  (it is probably documented on the mfg site of the touchscreen though).
 
The Intel based HD sound chip on the Openpeak motherboard is levels above what is on the RPi2 today. 
I use it today (modded box) - IE: optical out to MM in for one subzone.
 

 

[pete_c]

Relating to the Amazon Echo and integration with Homeseer I will most likely do an on demand thing whenever I want it utilized. 
 
I can now shut off the Amazon Echo internet tether via SSH event to the wireless AP from Homeseer or just a cron job on the wireless box.
 
IE:
 
root@ICS-Echo:~# wl radio off - disables echo internet connectivity
root@ICS-Echo:~# wl radio on - enables echo internet connectivity
 
root@ICS-Echo:~# wl assoclist - you can do a mac to DHCP thing or just validate the connectivity this way.
assoclist A0:02C:XX:XX:XX
 
Main mothership here has 16 serial port devices plugged in to it along with 7 USB devices along with some virtual devices on the network.  (thinking now I am at about 20 plus hardware connections).
 
My goal here is to turn the Amazon Echo in to a combo TTS/VR automation piece of hardware that will hang off the mothership. 
 
There will be no automation dependencies on it.  It will just be a tool to be able to use whenever.
 
Well similarly to the Kinect stuff.
 

 

 

[deandob]

Pete. Excellent pointers on the screens, thanks for sharing. I'll check them out. I am specifically looking for RGB24 modules but hacking older devices might be better especially if they have capacitive touch screens.

Back to the Echo, my Echo arrives tomorrow. It will augment the existing system not replace it so I'm hoping I can redirect VR commands to a URL on my hub and have the hub stream audio to the Echo. Lets see.

 

[pete_c]

Will look here at what I have.  Many years ago went to using / playing with those tiny '386 one board CPUs on modded bank terminals - well kiosks terminals (found a reference to them on the linux-hacker dot com site).  I was changing the LCD's over to touchscreens.  Interesting cuz a friend of mine that worked for GM did the tweaks to the LCD bios for making it work as a touchscreen.  I have all of the pieces here some place and will look.

 

[pete_c]

Here are a couple of You Tube videos from a Homeseer user that integrated the use of the Amazon Echo with Homeseer.  He is mixing it with MS SAPI Ivona (which is mentioned earlier).  Here also do have Ivona in my collection of TTS SAPI fonts.  Never pay much attention these days though.
 
Part 1
 
[youtube]http://youtu.be/JRhyGXXj0fs[/youtube]
 
Part 2
 
[youtube]http://youtu.be/nneSeHw2U_o[/youtube]

 

[wuench]

Just finished my CQC driver that simulates the hue.   No need for that java program any longer.   Here is a video of it in action... (I still get my coffee the old fasioned way)...
 

 

[pete_c]

Very nice Wuench!
 
Posted your video over on the Homeseer forum.