This is an example of a WordPress page, you could edit this to put information about yourself or your site so readers know where you are coming from. You can create as many pages like this one or sub-pages as you like and manage all of your content inside of WordPress.
Recent Comments
alx on Outdoors laptop (part 2) alx on Running your own mail ser… s3hh on Outdoors laptop (part 2) alx on Outdoors laptop (part 2) s3hh on Outdoors laptop (part 2)
S3hh's Blog 
Hello s3hh; I am trying to configure some KVM guests with static IPs to work with OVS. The guests egress fine, but I cannot reach them from the host or our intranet. Do you have a Quick ‘HowTo’ on setting the ingress to guest VMs on OVS? Any help is greatly appreciated.
I don’t, sorry. Anyone else?
Wow, what a highjacked thread. Anyone know the anser to my Dec 11th question?
If it’s not working from the host ,you should be able to fix that by following the example in /etc/init/lxc-net.conf. Not working from the intranet will require something more baroque. Other hosts on the intranet need to somehow be told to go through your host to get to the ovs bridge’s network. Unless you put all the other hosts on the intranet on the ovs bridge with gre tunnels as well.
Hi s3hh. I like your uvtools PPA. Simple and effective. I’m running it on Ubuntu 14.04 Beta 2 on an Intel I7 4th gen quadcore with 2 threads per core. All new VM guests start with only one CPU/core. I’d like to configure the guests from the command line (uvt-kvm create) to start with a specific cpu count, like 6. Alternately, could I use virsh to re-configure the guest once it starts?
Hi,
the ppa isn’t mine, nor is uvtools – credit for both goes to rbasak 🙂
In answer to your question, he told me “uvt-kvm create –cpu 6”. Good luck!
Thank you. Of course, this is a perfect case of RTFM. After reading your answer, I tried “man uvt-kvm”. Sure enough, many options popped up in the manual, including this one, memory and disk space. Thanks again!
Hi S3hh,
I’m trying to setup an SSH to my debain container but it’s running the same IP address on the host so it’s a failed attempt to do so. Your blog said LXC can’t be bridged wirelessly but you also said you need to forward the traffic so it could? Now, what do you mean by that?
I’m running network-manager to replace the network interface file, so naturally I can’t have both. What’s your suggestion?
You cannot put the wireless nic into a bridge and then add the container’s nic to the bridge.
So what you can do is to create an empty bridge, forward it’s traffic to the wireless nic, and bridge the container nics to the new bridge.
See the lxc-net.conf upstart job for an example: https://github.com/lxc/lxc/blob/master/config/init/upstart/lxc-net.conf
Thanks for the prompt reply,
The link is hard to understand since I’m not a programmer but it’s a script in Ubuntu that appears using iptables to direct the bridging port. I’m confused at what’s an ’empty’ bridge, do you mean where all the options (such as IP address, netmask, gateway..etc.) are omitted? This example came from: https://wiki.debian.org/LXC/SimpleBridge.
I get the idea of forwarding the traffic through the command that’s in your link: echo 1 > /proc/sys/net/ipv4/ip_forward.
So on the container side, you would do the same: install bridge-utils, configure the network interface to have identifical network variables (network.link, network.type..etc) and hope for the best?
If the answer is in some of your blogs which I’ve read your “LXC containers on a host with wireless”, then please forward me to it, if it saves you time.
An empty bridge is simply a bridge with no network interfaces attached yet. You can create a new one with
brctl addbr br1
ifconfig br1 0.0.0 up
The iptables rules masquerade traffic from any interfaces on the bridge (i.e. your containers) to your default route, presumably over your wireless nic.
Indeed, the “LXC containers on a host with wireless” did show a way to create such a forwarding bridge. It was written before our ubuntu lxc package did that setup for you.
Ok, so I would only need to add the new text you’ve inputted to the interface file (assuming bridge-utils & dnsmasq-base is installed). The blog shows that you’ve created a forwarding executable script that’s activiated when the bridge goes up via the ‘post-up’ variable but the setup you’ve shown me contain two different scripts of starting and stopping LXC, which results in creating/deleting a network_up file. Does the script in the blog “forward its traffic to the wireless NIC” and the setup scripts “bridges the container nics to the new bridge”? If so, then I would just use the post-up/post-down command in the interface file?
Yes the script both sets up the forwarding, and sets up a dnsmasq to listen on that bridge and hand out ip addresses to the containers, and there is a hunk which goes into /etc/network/interfaces, which creates the lxcbr0 bridge and then calls the script when it is up.
When you say “the setup you’ve shown me contain two different scripts of starting and stopping LXC”, I don’t know what you are referring to.
It’s nothing, I was referring to the post-stop and pre-start script (which are similar). Is there a difference between the blog you’ve posted versus the setup you’ve shown me? Because, it seems like the setup you shown me looks automate and if I install the package, it’ll do its job. That being the case, lxc-net.conf file needs be to created (on /etc/init/default directory) in my host and if I simply login into my container, the script will run?
EDIT: lxc-net.conf should be created on /etc/init.
Let’s do this hands on over what we’ve discussed, what I can deduce so far is that the lxc-net.conf has setup all the necessary steps described in your blog. So, focusing on that. I’ve created conf file in my directory described above, input variable USE_LXC_BRIDGE=”true” in my /etc/default/lxc, however I’m confused at what bridge should be put on LXC_BRIDGE. Is it the empty interface? Should I also create a bridge on the container as well?
LXC_BRIDGE=lxcbr0
Last week pitti offered a commit to pull out the core of lxc-net.conf into a standalone script, which you should be able to use. See https://lists.linuxcontainers.org/pipermail/lxc-devel/2014-July/009888.html
I see. So the script for init system uses different script options. Now I’m using the one for init script which uses the start and stop option. Since my config file for my container has no variables for the network, I’m unsure what to put in.
The defaults should be fine, creating an lxcbr0 with address 10.0.3.1.
The default for my config file just uses the hosts’ network stack so it’s empty of network variables but this is what I inputted:
lxc.network.type = veth
lxc.network.flags= up
lxc.network.link= lxcbr0
lxc.network.ipv4 = 10.0.3.1
Does this sound right?
Oh, I misunderstood – you’re asking about the container configuratoin file? Did you start a dnsmasq on lxcbr0 on the host? If so you can just do
lxc.network.type = veth
lxc.network.flags = up
lxc.network.link = lxcbr0
If you didn’t start a dnsmasq then you’ll need to specify an ipv4 address, but 10.0.3.1 is the bridge’s ip, so you’ll want to pick something else like 10.0.3.10.
Yes, I didn’t realized I haven’t installed dnsmasq package yet but instead chose its base. I’ve edited the conf file on uncommenting the interface option then input lxcbr0. The DHCP server starts as soon it was installed. To summarize, I’ve created an empty network interface you’ve described. Installed both bridge-utils & dnsmasq. Created the lxc-net.conf file on /etc/init directory as outlined on your link. Add USE_LXC_BRIDGE = ‘true’ to /etc/default/lxc file. The container’s config file includes the three network variables you’ve told me.
There seems to be a problem on LXC recognizing lxcbr0. What concerns me is that the empty interface doesn’t seem to play a role into this. It just adds a bridge name called ‘br1’, and an empty address.
Please note that I’m currently testing this setup on Debian.
Where does the name ‘br1’ come from? At this point how you are creating the bridge? Are you using Martin’s script?
br1 was the empty interface I added in the host interface file – sorry about the confusion. Yes, I used the init script from Martin. Does the script need to be edited to include routing traffic to my wireless NiC?
It was the empty interface you mentioned before – sorry about the confusion. Yes, I’m using Martin’s script you’ve provided that’s for init not upstart. Does the script need to be edited in order to route traffic to my wireless NIC?
No, the script should do all that is needed.
It appears that I’m having issues starting my container spawning an error message:
lxc-start: failed to attach ‘vethAqIist’ to the bridge ‘lxcbr0’ : No such device
Ifconfig shows that lxcbr0 indeed did not exist. I’m not sure what’s wrong but it seems that I may need to create lxcbr0 virtual interface on the host. Is that true?
Referring back to the script, I check on the /run directory and confirm there’s no lxc file there. Is this normal? I’m running version “0.8.0~rc1-8+deb7u2” of LXC provided under the Debian repository.
I’d expect the script to check for /sys/class/net/${LXC_BRIDGE}, not for /run/lxc. Indeed something has to do a ‘brctl addbr lxcbr0’ (or add lxcbr0 to /etc/network/interfaces) to make it exist.
I’m afraid I’m not familiar with the debian pkg, but I’ll look at it soon.
I confirmed that there’s no lxcbr0 in the /sys/class/net directory, adding the following to the host’s interface file didn’t create a new bridge that’s supposedly shown by ifconfig:
auto lxcbr0
iface lxcbr0 inet static
address 10.0.3.1
netmask 255.255.255.0
Any suggestions?
Are the lines after the first one indented?
Now do ‘ifup lxcbr0’. (that should happen automatically on reboot, though with network-manager installed it may not)
I’ve indented the lines after the first one, ifup said there’s no ‘lxcbr0’ device.
After doing some solo research, the missing variable bridge_ports triggered an reaction from ifconfig. If I used the same static setup as above then it’ll say it already has been configured, DHCP produce a DHCPDiscover pursuit but it results in no connection. Now I’ve tested that interface lxcbr0 has been configured whether or not I set it to DHCP or static.
The container is running, however lxcbr0 doesn’t seem to have an IP address assigned.
The container doesn’t seem to have lxcbr0 bridge configured. Is this normal?
lxcbr0 doesn’t exist in the container unless you’re going to create nested containers inside that container. The container will simply have an eth0, which is a veth tunnel endpoint whose other end is bridged to the host’s lxcbr0.
I found another way using libvirt to do this.
Thanks for your continual assistance and advice.
Hello,
I have problem with LXC configuration. I created shared mount point (mount bind mount share) between host and several other containers and I want if one container mount sda1 device than all other container will see also this mount and also host will be able to see it.
This works with LXC 0.7.5 but stop works with 0.9.0 and a later.
Can you help me, please?
Best regards,
Lukasz
What configuration did you use with 0.7.5 to make it work?
0.9.0 is still quite old, we’re on 1.1.4 right now… but fundamentally nothing should have changed here – depending on the configuration you are using.
Thank you very much for fast feedback.
Configuration is very simple and the problem can be related with the fact that I am not using lxc.rootfs option and I can not use it.
After run container without changing rootfs I make several things and then change rootfs dir with chroot.
A simple way to reproduce the problem:
———————————————————
Run the following command from the first console:
$ cd /tmp
$ mkdir shared1
$ mkdir shared2
$ mount -t tmpfs tmpfs /tmp/shared1
$ mount –bind /tmp/shared1 /tmp/shared1
$ mount –make-shared /tmp/shared1
$ mount –bind /tmp/shared1 /tmp/shared2
$ touch /tmp/empty.conf
$ LD_LIBRARY_PATH=/mnt/new2/lxc/lxc-0.9.0/src/lxc /mnt/new2/lxc/lxc-0.9.0/src/lxc/lxc-start -n TEST -f /tmp/empty.conf /bin/sh
$ mkdir /tmp/shared2/mountpoint
$ mount -t tmpfs tmpfs /tmp/shared2/mountpoint/
$ touch /tmp/shared2/mountpoint/test.txt
Now from the host (second console) check if you have access to the “/tmp/shared2/mountpoint/test.txt” or “/tmp/shared/mountpoint/test.txt”
$ ls -l /tmp/shared2/
total 0
drwxr-xr-x 2 root root 40 Oct 16 00:37 mountpoint
$ ls -l /tmp/shared2/mountpoint/test.txt
ls: cannot access /tmp/shared/mountpoint/test.txt: No such file or directory
Best regards,
Lukasz
Sorry, inside container I mount real block device not tmpfs.
$ mount -t vfat /dev/sda1 /tmp/shared2/mountpoint/
Sorry for the mistake.
Are there any chances for help, please?
A few things,
1. you’d probably want to use rshared rather than just shared
2. likewise rbind rather than bind
3. lxc makes the container’s rootfs MS_SLAVE. You’ll need to actually bind the MS_SHARED directory into the container using a lxc.mount.entry in the container configuration.
No. I want use shared for bidirectional share.
However, even if I try your suggestion nothing change…
Adding lxc.mount.entry = /tmp/shared1 /tmp/shared2 none bind 0 0
or
lxc.mount.entry = /tmp/shared1 /tmp/shared2 none rbind 0 0
also not help.
Do you have an idea why it works with LXC 0.7.5 and stop working and stopped working in later versions? What is the root cause?
OK. So, as I understand the problem is here:
static int setup_rootfs(struct lxc_conf *conf)
{
const struct lxc_rootfs *rootfs = &conf->rootfs;
if (!rootfs->path) {
if (mount(“”, “/”, NULL, MS_SLAVE|MS_REC, 0)) {
SYSERROR(“Failed to make / rslave”);
return -1;
}
return 0;
}
….
Hmm, but what should I do when I want to have bidirectional share?
For example I have container A, B and C and Host and I want that no matter who make mount everyone will have access to mounted fs.
Is this possible? Just as it was possible in LXC 0.7.5.
Yes that’s why it used to work for you the way you show. But you should still be able to make it work using lxc.mount.entry.
Hm. Yeah the problem is that happens after we have done the lxc.mount.entry’s.
So what we really want is a way to tell lxc to keep certain mounts shared. (We can’t just do it by default because of things like /var/run/netns)
OK. So, without source code modification it will be not possible?
I am really grateful for your help. Without information “lxc makes the container’s rootfs MS_SLAVE” I would not know what’s going on 🙂
Hi,
right, you can set up a MS_SLAVE but not MS_SHARED mount between the two. So to work around it, you can make a MS_SHARED mount, start the container, then have the host mount a MS_SHARED directory under the MS_SLAVE one. I.e. if /mnt/containers/slave/c1 on the host is bound to /shared in the container, then have /mnt/containers/shared/c1 mounted into the container under /shared/host/. Now /shared/host/ will be MS_SHARED.
But how…, can you help me?
The entry:
lxc.mount.entry = /tmp/shared1 /tmp/shared2 none bind 0 0
nor
lxc.mount.entry = /tmp/shared1 /tmp/shared2 none rbind 0 0
unfortunately not working.
lxc-start read this line because when I made mistake they lxc-start failed.
@s3hh
Thank you very much for your help.
I hope that I can ever repay.
Regards,
Lukasz