[NetBehaviour] YandexBot

[Alan Sondheim]

Question - my site is commercially-operated and this month for example I
ran 321 Gigabytes of hits etc. - almost a third of a terabyte. I get
complete stats on this and I found that YandexBot from Russia has created
the majority of the bandwidth by a long shot -
YandexBot 24,212+502 255.00 GB 30 Nov 2019 - 06:58
That's "hits" "bandwidth" and "last visit" -
This comes in the midst of for example 39,466 useful hits (i.e. accesses to
my site) -
Has anyone dealt with this before? Online I found
>From this url -
https://yandex.com/support/webmaster/robot-workings/check-yandex-robots.html
There's this -
How to check that a robot belongs to Yandex

Some robots can disguise themselves as Yandex robots by indicating the
relevant User Agent. You can check the authenticity of a robot using
reverse DNS lookup.

Just follow these steps:

   1. Determine the IP address of the user agent in question
   <https://yandex.com/support/webmaster/robot-workings/check-yandex-robots.html#robot-in-logs>
   using your server logs.
   2. Use a reverse DNS lookup of the IP address to determine the host
   domain name.
   3. Check whether the host belongs to Yandex. All Yandex robot names end
   in yandex.ru, yandex.net or yandex.com. If the host name has a different
   ending, the robot does not belong to Yandex.
   4. Make sure that the name is correct. Use a forward DNS lookup to get
   the IP address corresponding to the host name. It should match the IP
   address used in the reverse DNS lookup. If the IP addresses do not match it
   means that the host name is fake.

--- Has any else run into this? Why such enormous bandwidth? (My site is
max 20 gb.) Has anyone else dealt with Yandex?
Thanks, Alan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.netbehaviour.org/pipermail/netbehaviour/attachments/20191130/677070f0/attachment.htm>