Appliance to monitor internet traffic

Ira

Active Member
Just found out I'm a "victim" of Xfinity's data usage limit, although it's my first time so I shouldn't get charged. We consistently use 250-300 GB each month on a plan that has a 1.2TB limit. Three weeks into February and their stats show we've used 1.126 TB so far this month. Nothing has changed here that would warrant a four-fold increase, so I'm pretty sure the stats are wrong. However, there are a lot of posts on the Xfinity forum of the same scenario, and Xfinity never does anything about it, other than refusing to back out the excess data charge.

I have an Xfinity xfi router/modem with wifi turned off, because I have an Asus wifi router running in AP mode for my wireless devices. The Asus device and all other wired devices (PC's, other switches, printers, network drives, etc.) ultimately connect to the xfi router via a single dumb switch.

Is there an appliance that I can put between the xfi router and the dumb switch connected to it (or replace the dumb switch with the appliance) that will allow me to monitor internet traffic to/from my LAN? I don't care about local traffic. I just want to be able to see how much internet data I'm using by device. I'm not interested in a software solution, or repurposing a switch with other software.

Thanks,
Ira
 
About two years ago I purchased a Fortigate Next Generation Firewall, and its been life changing. A 40F which works fine for my house that has about 50 devices, half of Wi-Fi. Long-story-short, after a year+ of learning, it gives you incredible insight into what is coming in and what is going out at a fine level. I will tell you, ads and spying add at least 35% to your traffic, and there is SO much more trash than you could imagine. Go to CNN.com and your computer immediately connects with over 150 web sites sending and getting data. You will also see, at least every 5 minutes, some hacker attempting to use some exploit to get into your system.

Yes, this is the Cadillac of systems, along with similar systems, and there are many "simpler" systems that can get you summaries, but just looking at the summaries doesn't give you the full picture. Like Facebook and TikToc had large amounts of data from my house, but guess what, my wife or I have never been on these and we don't have accounts, but they still spy on you.

So long answer is, there isn't any "easy" solution to give you what you want, or if you think you find one, it will only give you a partial answer. But if you really dig in with the right tools, you'll find many many incredible things.

But to give you an idea, in my house, other than ads and spying, streaming TV is 90% of the data, computers (Macs) are 6%, phones are 2% and maybe 2% for everything else, and 4K uses 4 times more than HD. So, any new 4K TVs lately?? I have my 4K TV running at HD because its so much less data, and honestly, I can't tell the difference on my TV.
 
Free:
I'd start with implementing something like pi-hole, which can block a good chunk of ad traffic, but also give you some insights into your internet traffic. You could even run this in a VM for a short period of time just to get this data. It's all DNS based, so anything configured to use DNS over HTTPS etc would be able to get around this.

$$
pfSense/OPNSense: there are some bandwidth add-ons, but it's very basic. The software is free, but you either need to get their appliance version, or install it on an older PC/VM to keep cost down.

$$$:
A gateway/firewall which offers traffic analytics such as the Unifi Dream Machine series (not sure if you need the Pro version for better analytics), or the above-mentioned Fortigate 40F, which is definitely a great approach, but the annual subscription is a killer :( Wish they'd offer a 'home' version, would love to get my 40F going again.
 
I looked at UTM appliances about eight years ago that were similar to the Fortigate. At the time, I was interested in the security features, mainly wanting to consolidate security in a single appliance, but like @electron, I was turned off by the annual subscription fees.

@ano what subscriptions do you have? I played around with the demo on their website, but it's a lot to take in. I don't have any idea what subscriptions would be needed for a reasonable home network solution that includes a reasonable amount of security and the ability to monitor and track data usage over periods of time, e.g., what devices used the most data over the last month and when did they use it. Does the 40F become a dumb network switch if you don't have any subscriptions?

@pete_c I read a bunch of the threads on the Xfinity form earlier today. Unfortunately, I didn't see any where Xfinity agreed with the customer.

Unfortunately, they are the only game in town at the new house location for anything over a couple of Mbps internet speeds. I get one Gbps with Xfinity at my current home, and it's surprisingly reliable. I hate to do it, but I will probably upgrade to unlimited data since it only costs me an additional $10/month (upgrade from Xfi Standard to Xfi Complete).

I spent hours this week trying to get them to send out an installation technician to our house that is under construction so I can find out what to do for the proper rough-in for their service. Even though we are a customer at our current home, they couldn't send a technician to the new house location because we don't have service there. Their solution... open a new service installation ticket on the new address, then cancel it after the technician comes out.
 
$$$:
A gateway/firewall which offers traffic analytics such as the Unifi Dream Machine series (not sure if you need the Pro version for better analytics), or the above-mentioned Fortigate 40F, which is definitely a great approach, but the annual subscription is a killer :( Wish they'd offer a 'home' version, would love to get my 40F going again.
Subscription is about $350/year, but you get ALOT for that. The virus updates occur every 4 hours, and their web and application databases have millions of entries. But in any case, you don't need to renew the updates if you don't want, and it will continue to run fine. Everything is local, its NOT web based, it just won't have updates.

PiHoles are fine, and I used one, but that and browser ad blockers only block some of the web page ads you see. There is SO much more than that, and many ads have the direct IP address, so don't even use DNS. And much of the spying occurs on Fire TVs, Alexas, phones, smart TVs, and more. Did you know most smart TVs now spy on your video and recognize it when you just feed in an HDMI input. There is over 100Mb "stuff" from my Samsung TV everyday and I only use it as a monitor, never a TV. So they are selling to use your data, while they use your data which YOU pay for!!!
 
@ano is your subscription...

FortiGate-40F 1 Year SMB Protection (IPS, Advanced Malware Protection, Application Control, URL, DNS & Video Filtering, Antispam, plus FortiGate Cloud subscription and FortiCare Premium)

...or...

FortiGate-40F 1 Year Unified Threat Protection (UTP) (IPS, Advanced Malware Protection, Application Control, URL, DNS & Video Filtering, Antispam Service, and FortiCare Premium)

...or something else? Does the subscription you have give you the ability to see data usage by device over time (the main requirement I have right now) without any additional tools? Will it tell me the internet data usage for my PC for the last 30 days? If so, I may pull the trigger on one of these.

Are these good enough such that I could get rid of endpoint software, e.g., Norton, and just use regular Windows 10 security with the NGFW? That would save me $100/year.

Up to a few times a week for work, I have to download files that can be 40GB. Will the 40F have a noticeable effect on download speeds because it's inspecting all of that? If so, can security for downloading those files be turned off since they are coming from a trusted source thru a VPN connection?

Thanks,
Ira
 
What you can do is request a ticket to be opened on the XFinity forum to monitor the volume of traffic and concurrently use one of the tools on DSL Reports to do the same and prove up your actual usage.

or

Just pay the extra $10 per month and not ask any questions.
 
I found the culprit. A few weeks ago, I changed email clients from eM Client to Outlook because eM client wasn't playing well with my AOL mail account (which is my primary account). I had switched to eM Client from Thunderbird a couple of months earlier because the latest release of Thunderbird is full of problems (I had been using TBird for at least 10 years).

I have about 27K emails in my AOL inbox, and about 8K in my AOL sent folder. I noticed that several times a day, Outlook would have only/exactly 1000 emails in my Inbox for my AOL account. It would then start resynchronizing and after a few minutes, show the correct 27K emails again. Same thing happens with the AOL sent folder, but it doesn't seem to happen as often. This is a well known issue and seems to affect a lot of email clients using AOL IMAP connections. From what I can tell, there is no solution, and AOL always blames the client software, and the client software always blames AOL. For this reason, I had already planned to stop using my AOL account as my primary email and transition to something else (although not sure what to transition to yet).

The Task Manager App History shows 430 GB of network traffic in the last 30 days for Outlook. It looks like I started using outlook right around the first of February.

Still would like a way to monitor and report on internet traffic by device.
 
What you are seeing in AOL Email is the result of combining AOL with Yahoo mail. Way back here had Verizon Email and XFinity Email and both appeared to be a dumping ground for spam emails with the inability to shut off the spam emails (sort of a deliberate effort).

Internet Oldies AOL And Yahoo Are Sold ... Again

They want to make money now.


The Task Manager App History shows 430 GB of network traffic in the last 30 days for Outlook. It looks like I started using outlook right around the first of February.

Lately same thing has happened with free Outlook mail with mostly advertisements to purchase Microsoft 360. I have 3 Outlook accounts here and only the "free" Outlook is getting spammed...almost every day lately...might be also related to me using Microsoft Windows 11 (sometimes versus Ubuntu Linux).

Microsoft's Outlook: Cloudy with a chance of junk-mail-stuffed inboxes

Google Mail is still working fine with spam filtering.

My email associated with my domain name was working well until it was switched ("free") to Microsoft paid Outlook with same domain name until I received first invoice / billing which was 4 times the amount of the email service that I was using (20 years now). I changed it back to another domain name email (same company). Customer service asked me how I found out about the service and I mentioned that it was on the domain service web page.

So today in 2023 see my ".edu" Microsoft 360 Outlook account works the best and is free provided by my alma mater. (alma mater from before the internet and email days). I get no spam at all on that account.

I also have a Proton Email account which I do not really use much but have read good things about it. (for paid and free email accounts).
 
Last edited:
The more I look at the network stats I have for my PC, I can see that Outlook (with the AOL issue) is the consumer of a lot of the data usage increase. However, it doesn't explain all of it. If I take the high side of my previous monthly average for all devices on my network (300GB), and add all of the Outlook data usage on my PC (436 GB), I'm at 736GB for the month of February. Xfinity says I've used 1.17TB so far in February. So there's about 434GB that I still can't explain.

Outlook is responsible for 434GB out of the total 470GB of network data usage on my PC.
 
Subscription is about $350/year, but you get ALOT for that. The virus updates occur every 4 hours, and their web and application databases have millions of entries. But in any case, you don't need to renew the updates if you don't want, and it will continue to run fine. Everything is local, its NOT web based, it just won't have updates.

PiHoles are fine, and I used one, but that and browser ad blockers only block some of the web page ads you see. There is SO much more than that, and many ads have the direct IP address, so don't even use DNS. And much of the spying occurs on Fire TVs, Alexas, phones, smart TVs, and more. Did you know most smart TVs now spy on your video and recognize it when you just feed in an HDMI input. There is over 100Mb "stuff" from my Samsung TV everyday and I only use it as a monitor, never a TV. So they are selling to use your data, while they use your data which YOU pay for!!!

Oh I understand the risks, I'm in the industry. Most of my network devices are restricted via isolated VLANs, while also avoiding devices which require any type of internet connectivity as much as possible. My Roku players are the only exception, but a good chunk of that traffic is dropped by my pihole setup. As I said before, DNS over HTTPS is making the Pi-hole approach less effective, but just to give you an idea, here are my stats as of right now:

Over the last 24 hours, there were 91,596 total queries, 44,570 were blocked (48%) with over 223,000 domains on the blocklist. Biggest offenders are my Roku devices and Microsoft Windows.

The problem with the Fortigate is that once the support ends, you aren't getting updates, including security updates. Firewalls which aren't receiving security updates anymore are just as big of a risk. Just today NSA announced that you shouldn't rely on your ISP provided router for just this reason. Like I said, big fan over the Fortigate, just out of budget for the typical home user (ignoring the learning curve issue).
 
So here looked at my data usage on one of two accounts. Difficult to find data usage as I see nothing on the monthly bills and see a jump in January...which doesn't make any sense to me....that it went back down in February.

I do not like what I see as I know that my usage of the Internet hasn't changed at all in the last year.

data.jpg

I agree with @electron relating to using the Pi-Hole and separate VLANs. That said there is only two of us here and wife only shops on the Internet.

Here utilize PFSense + with its available tools and the tools do work according to stats that I see.

PFSense packages

One great little tool is called PFBlockerNG

Utility for controlling connections through the firewall based on more general criteria than firewall rules (e.g. by country, by domain name, etc). Manages IPv4/v6 List Sources into ‘Deny, Permit or Match’ formats. GeoIP database by MaxMind Inc. (GeoLite2 Free version). De-Duplication, Suppression, and Reputation enhancements. Provision to download from diverse List formats. Advanced Integration for Proofpoint ET IQRisk IP Reputation Threat Sources. Domain Name (DNSBL) blocking via Unbound DNS Resolver.

Here shut off the Roku devices and mostly use Ubuntu desktop (dual booting Windows 11 / Ubuntu) and use Windows 11 because my Wife prefers it over Linux. I use Linux (not Android or Windows) Kodi as STB devices. You need a subscription to stream Amazon on demand (or Netflix et al) BUT you are presented with a movie list and no pop ups.

Recently helped a peer (with PFSense) do what @electron do with his home network. (IE: isolating his iOT devices to a VLAN et al). He did get me to purchase L2/L3 switches recently (where I was doing only L2 managed switches beforehand). He was getting usage overage messages on his XFinity invoices every month for a period of time last year. He is Texas / Houston and XFinity doubled his Internet speeds sans any charges which I guess is a nice to have.

Here is the data usage of peer near Houston, TX.

DL-Data0.jpg
DL-Data.jpg


I did mention to him to document and not over complicate his home network as he provided me a list of his VLANs / firewall rules per network et al. Here if I pass away my wife will just unplug every network device on line.

I still believe that XFinity has gone a tad over zealous with their Internet usage stats as the complaints on the XFinity forum are all the same. That or they have become granular tweaked their monitoring using new tools...sort of like (bad analogy) when there was a push to put traffic cams all over the place to increase revenues for a towns et al.
 
Last edited:
@ano is your subscription...

FortiGate-40F 1 Year SMB Protection (IPS, Advanced Malware Protection, Application Control, URL, DNS & Video Filtering, Antispam, plus FortiGate Cloud subscription and FortiCare Premium)

...or...

FortiGate-40F 1 Year Unified Threat Protection (UTP) (IPS, Advanced Malware Protection, Application Control, URL, DNS & Video Filtering, Antispam Service, and FortiCare Premium)
If you just want to track data per device, you don't need any extra services. They identify viruses, applications, bad web sites, etc. All you would need to do is make a firewall rule for each device. So get the mac address and label each device.

Screenshot .jpg
The subscription would help you identify each packet of data. But it does take some experience to set it all up. When you buy it you get a blank slate. I've grouped things like Macs, Phones, Watches, but you don't have to.
 
The problem with the Fortigate is that once the support ends, you aren't getting updates, including security updates. Firewalls which aren't receiving security updates anymore are just as big of a risk. Just today NSA announced that you shouldn't rely on your ISP provided router for just this reason. Like I said, big fan over the Fortigate, just out of budget for the typical home user (ignoring the learning curve issue).
So do you realize this, lets say you go to a web site, xyz.com, but you don't worry because you have the ISP router with "virus protection" so your all set, right?? Ha Ha. If that web site is secure, like most are today, then from the browser to the web site are encrypted (that little lock symbol) and guess what, your router CAN'T SEE ANY OF THAT DATA. So you better hope that none of xyz.com advertisers, or xyz.com itself is not infected, because guess what, you are now infected.

On a Fortigate, you install a certificate from the router on all your computers, or important devices, at least, and the Fortigate decrypts the encryption, scans it, that reencrypts the data so your browser puts on the lock, and everything is encrypted end-to-end, but it is also fully scanned. Regular routers don't do that. Same with encrypted DNS traffic (TLS over port 853 is better encryption than DNS over HTTPS, by the way.). And the Fortigate intercepts the DNS traffic, even if it doesn't use the DNS server you gave it. (Many use 8.8.8.8 no matter what you TOLD it to use.) And a good bit of traffic NEVER goes to a DNS server at all. The IP address is hard coded in it. If the Fortigate can't decrypt it for whatever reason, its blocked.

You can check any web address here, and they have millions. Don't forget to check your not a robot.
https://www.fortiguard.com/webfilter
 
Back
Top