Welcome, Guest. Please login or register.

Author Topic: Introduction to Project "Kharon"  (Read 2564 times)

0 Members and 1 Guest are viewing this topic.

Offline Come-from-Beyond

  • Moderator
  • Legendary Nxter
  • *****
  • Posts: 603
    • View Profile
Introduction to Project "Kharon"
« on: December 17, 2013, 08:42:25 PM »
From the very first days of Nxt we faced constant attacks from zombie nodes that were trying to disrupt normal functioning of Nxt network. These attacks were clearly identified and easily counteracted, but in the future we will see much more sophisticated attacks conducted by powerful entities. One of such attacks will be application-level DDoS attacks which are hard to avoid. The main goal of project "Kharon" is development of a filtering system that is supposed to cut off inbound request and responses sent by adversaries.

We propose to create a system that will use artificial neural networks to filter the traffic. This is a long-term project, before implementing it in software we have to solve the following problems:

- What metrics to use as input values
- What a structure to choose for the neural network
- What algorithms to use for its teaching
- and much more

This sub-forum is dedicated to discussion of all aspects of the project. Feel free to contribute with ideas and constructive criticism.
« Last Edit: December 17, 2013, 08:48:55 PM by Come-from-Beyond »

Offline joefox

  • Senior Nxter
  • ****
  • Posts: 393
    • View Profile
    • The Nxt Wiki
Re: Introduction to Project "Kharon"
« Reply #1 on: December 17, 2013, 09:14:35 PM »
Thanks for getting this started!

You're working hard on the software, so you have a much better understanding of the methods people are using to attempt DDoS and other attacks.  What are the "signatures" for these, in your opinion?  I'd love to toss in some brainstorming but I think I need more background on the kinds of attacks the network is seeing.
Administrator of the Nxt Wiki at  http://wiki.nxtcrypto.org/

Like my advice OR the wiki? Please donate to account 1234567740944417915

Offline landomata

  • Senior Nxter
  • ****
  • Posts: 393
    • View Profile
Re: Introduction to Project "Kharon"
« Reply #2 on: December 17, 2013, 09:23:39 PM »
I am not a machine learning expert....i;m actually very ignorant but would it hurt to contact the below gentlemen he is a very nice guy & im sure he would love to help this project develop with his expert advice:

Geoffrey Hinton 
[email protected]


Offline Come-from-Beyond

  • Moderator
  • Legendary Nxter
  • *****
  • Posts: 603
    • View Profile
Re: Introduction to Project "Kharon"
« Reply #3 on: December 17, 2013, 10:25:09 PM »
You're working hard on the software, so you have a much better understanding of the methods people are using to attempt DDoS and other attacks.  What are the "signatures" for these, in your opinion?  I'd love to toss in some brainstorming but I think I need more background on the kinds of attacks the network is seeing.

If that was possible to determine "signatures" then we would not need neural networks. The most interesting part in NN is that they see these signatures but noone, even creators, can figure out how they do that. A working network with a non-trivial structure is a black box.

Offline Come-from-Beyond

  • Moderator
  • Legendary Nxter
  • *****
  • Posts: 603
    • View Profile
Re: Introduction to Project "Kharon"
« Reply #4 on: December 17, 2013, 10:26:13 PM »
I am not a machine learning expert....i;m actually very ignorant but would it hurt to contact the below gentlemen he is a very nice guy & im sure he would love to help this project develop with his expert advice:

Geoffrey Hinton 
[email protected]


Thank u for the suggestion but I think our issue is not so interesting for him.

Offline iruu

  • Fresh Nxter
  • *
  • Posts: 48
    • View Profile
Re: Introduction to Project "Kharon"
« Reply #5 on: December 17, 2013, 10:36:23 PM »
This is a very bad idea. 
No matter how good the algorithm, there's always going to be a false positive. In this case, false positive means someone with legitimate account is going to be unable to use the currency, or is going to have great problems with it. 
False positives are acceptable in centralized systems, because if false positives are sufficiently rare, there's always some authority who can manually allow someone. No such thing here. 

Also, it's not a problem now. No point wasting time fighting an imagined problem, there's always something real to improve. 

If you absolutely must execute this bad idea, don't use neural networks. They're the most known, but very hard to get right, they're very prone to overfitting, and choosing the right structure is often pure guesswork.   
Use random forests. They're drastically easier to train, prediction is much faster and completely parallel, and results are very often much better than those from neural networks.
For extra accuracy, use also SVM (support vector machine) and combine the results of the two classifiers into something called an ensemble classifier. Ensembles made of many algorithms are the current state of the art.
54.227.95.223 - 24/7 nxt node | bitfinex referrer code - IiDoeUOa3i

Offline Come-from-Beyond

  • Moderator
  • Legendary Nxter
  • *****
  • Posts: 603
    • View Profile
Re: Introduction to Project "Kharon"
« Reply #6 on: December 17, 2013, 10:45:25 PM »
This is a very bad idea. 
No matter how good the algorithm, there's always going to be a false positive. In this case, false positive means someone with legitimate account is going to be unable to use the currency, or is going to have great problems with it. 
False positives are acceptable in centralized systems, because if false positives are sufficiently rare, there's always some authority who can manually allow someone. No such thing here. 

Also, it's not a problem now. No point wasting time fighting an imagined problem, there's always something real to improve. 

If you absolutely must execute this bad idea, don't use neural networks. They're the most known, but very hard to get right, they're very prone to overfitting, and choosing the right structure is often pure guesswork.   
Use random forests. They're drastically easier to train, prediction is much faster and completely parallel, and results are very often much better than those from neural networks.
For extra accuracy, use also SVM (support vector machine) and combine the results of the two classifiers into something called an ensemble classifier. Ensembles made of many algorithms are the current state of the art.

We r not going to block access for users, this is for peers only.

Maybe u r right and random forests will be better. But we have to make sure that NNs r not able to solve the problem, actually, noone knows that the forests can do that.

PS: It will be too late when the problem arises. We have to solve it in advance.

Offline iruu

  • Fresh Nxter
  • *
  • Posts: 48
    • View Profile
Re: Introduction to Project "Kharon"
« Reply #7 on: December 17, 2013, 10:51:53 PM »
But we have to make sure that NNs r not able to solve the problem, actually, noone knows that the forests can do that.

???
I don't understand the logic in your post. Why are NNs inherently preferable?
54.227.95.223 - 24/7 nxt node | bitfinex referrer code - IiDoeUOa3i

Offline Come-from-Beyond

  • Moderator
  • Legendary Nxter
  • *****
  • Posts: 603
    • View Profile
Re: Introduction to Project "Kharon"
« Reply #8 on: December 17, 2013, 11:06:54 PM »
But we have to make sure that NNs r not able to solve the problem, actually, noone knows that the forests can do that.

???
I don't understand the logic in your post. Why are NNs inherently preferable?

They r similar to our brains. My brain recognizes digits, but a forest near my house doesn't. :)

We could go both the ways, I'm familiar with NNs so I'll check them in the 1st place.

Offline utopianfuture

  • Senior Nxter
  • ****
  • Posts: 308
  • NXT ambassador
    • View Profile
Re: Introduction to Project "Kharon"
« Reply #9 on: December 17, 2013, 11:16:31 PM »
@Come from Beyond, I remember reading a comparison between POW and POS a while ago and a criticism against POS is that it costs almost nothing to mine (or forge) in POS and that's the reason POS is more prone to attack such as DOS. Meanwhile for POW, it is very costly to successfully mine a block and that is a financial barrier an attacker must overcome to harm the network. Can we make it that only account with a certain amount of balance can forge ? 
NXT acc 3041433146235555849. Donation welcome if you find me being helpful.

Offline mripper

  • Nxter
  • **
  • Posts: 85
    • View Profile
Re: Introduction to Project "Kharon"
« Reply #10 on: December 18, 2013, 07:12:18 AM »
@Come from Beyond, I remember reading a comparison between POW and POS a while ago and a criticism against POS is that it costs almost nothing to mine (or forge) in POS and that's the reason POS is more prone to attack such as DOS. Meanwhile for POW, it is very costly to successfully mine a block and that is a financial barrier an attacker must overcome to harm the network. Can we make it that only account with a certain amount of balance can forge ?

Yes, you have to have at least 1 Nxt to forge, however, the problem proposed by CFB isn't mining related, it's network and client related.
o7d9uocau4uuuan9e5digksr79b6l6sqslkb8u64mqfqq5843fbq4qfh87hik800g84v0um5cvtvtmoh0af1647rhfl76f9mv5jsor1mnq70te8n08q6ahhqejr1hukibjvmer5kocgbj50r5d2f7gt8lkpvr0s6

Offline Come-from-Beyond

  • Moderator
  • Legendary Nxter
  • *****
  • Posts: 603
    • View Profile
Re: Introduction to Project "Kharon"
« Reply #11 on: December 18, 2013, 09:00:49 AM »
@Come from Beyond, I remember reading a comparison between POW and POS a while ago and a criticism against POS is that it costs almost nothing to mine (or forge) in POS and that's the reason POS is more prone to attack such as DOS. Meanwhile for POW, it is very costly to successfully mine a block and that is a financial barrier an attacker must overcome to harm the network. Can we make it that only account with a certain amount of balance can forge ?

U confuse a network as links between nodes with a network as payment processing entity.

Offline smartwart

  • Fresh Nxter
  • *
  • Posts: 19
    • View Profile
Re: Introduction to Project "Kharon"
« Reply #12 on: December 18, 2013, 01:56:45 PM »
I agree with iruu that there are many things to solve in 1st place for the core development crew.
But the point is that the network security will become an issue sooner than later!!!
This must be taken into account as early as possible and become integrated.

I understand this topic as a call to the comunity to have a constructive discussion...
(not only to one direction, for sure)
And maybe the are some skilled, who are able to help?

From my own projects, I know this open source library which is well supported.
http://www.heatonresearch.com/

Also ther is a Java source...
Maybe it helps?
NxT: 13574045486980287597
-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
VPS NxT-Node Donation: 6207322156989043489

Offline rlh

  • Fresh Nxter
  • *
  • Posts: 24
    • View Profile
Re: Introduction to Project "Kharon"
« Reply #13 on: December 18, 2013, 08:33:03 PM »
I'm a developer by trade, but not a network/web developer.  However, from my observation, DDoS protection for P2P networks isn't this difficult if the P2P network is large and well distributed.

From what I understand, other crypto-coins that have simply implemented IP filtering have managed to mitigate most DDoS issues, so long as the coin's network was larger than the attacking DDoS network.

I recommend that each node black-list offending IPs for, say 8-12 hours, after receiving some many requests a minute.

Offline Come-from-Beyond

  • Moderator
  • Legendary Nxter
  • *****
  • Posts: 603
    • View Profile
Re: Introduction to Project "Kharon"
« Reply #14 on: December 18, 2013, 09:15:18 PM »
I'm a developer by trade, but not a network/web developer.  However, from my observation, DDoS protection for P2P networks isn't this difficult if the P2P network is large and well distributed.

From what I understand, other crypto-coins that have simply implemented IP filtering have managed to mitigate most DDoS issues, so long as the coin's network was larger than the attacking DDoS network.

I recommend that each node black-list offending IPs for, say 8-12 hours, after receiving some many requests a minute.

This is true for low-level DDoS attacks that r effectively blocked by tricks like syn-cookies. Application-level attacks r much worse, sometimes 1 computer is enough to overcome 1000s nodes.

Dumb blacklisting can lead to a network fragmentation making Nxt vulnerable to other types of attacks.