Wireless mesh networking has emerged as a key technology to provide

wide-coverage broadband networking. It benefits both service

providers with low cost in network deployment, and end users with

ubiquitous access to the Internet from anywhere at anytime. Wireless

mesh networks are vulnerable to malicious attacks due to the nature

of wireless communication and the lack of centralized network

infrastructure. Meanwhile, the capacity of multi-radio multi-channel

communication, the need for heterogeneous network integration, and

the demand for multi-hop wireless communication often make

traditional security mechanisms inefficient or infeasible.

Therefore, wireless mesh networks pose new challenges and call for

more effective and applicable solutions.

In this work, we identify the requirement for a systematic security

framework to protect wireless mesh networks and provide a security

system with heterogeneity-aware intrusion prevention mechanism,

cross-layer based intrusion detection technique, and a generic

intrusion response model.

Our major contributions lie in the following: (1) We identify the

architecture heterogeneity of wireless mesh networks and proposed a

novel heterogeneity-aware group key management framework which

combines the logical key hierarchical technique together with the

localized threshold-based technique. (2) To leverage link-aware

routing characteristics, we present a cross-layer based anomaly

detection model which utilizes machine learning algorithms for

profile training and intrusion detection. (3) We address the

automatic intrusion response problem in wireless mesh network by

providing a generic response model to describe the dependency of

system services and resources. The dependency graph is later used

for damage cost assessment and response cost evaluation. (4) We

build a wireless mesh network testbed and implemented a system

prototype for intrusion detection system. Our simulation and

experiment results show that our solutions outperform existing ones

and are practical for wireless mesh networks in terms of

communication overhead and performance speed.