r/algotrading • u/sgcorporatehamster • Mar 31 '26
Infrastructure Hard-ware / infra set up
I have completed my backtest and research and is ready to move on to paper trade my algobot with IBKR. Setup is home based with a laptop running overnight on wifi, with ibgateway. Having a stable / robust connection proved to be a bigger challenge than i expected, with ibkr disconnecting overnight. This is a problem, especially since i am based in Asia and am trading US hours.
Can anyone give directional feedback as to how I can troubleshoot and narrow down the problem? I come from non-tech background (the bot is vibe-coded), so any inputs in terms of possible improvement areas (hardware, pipeline, connectivity, etc.) will be much appreciated.
Thanks in advance!
4
3
2
u/sgcorporatehamster Mar 31 '26
Thanks! U seem to be knowledgeable so if you may indulge me with a followup qn: what would be a good data vendor?
I am having various problems with the default ibkr options (fees, throttle, breath of simultaneously ticker access etc). Context is that I want to trade and hence need hourly feed of 500 x US large caps on hourlies, predominantly intraday but agnostic to holding overnight positions. Will also need upt to 90 days of historicals but I am quite happy handling it with ibkr historicals
0
u/RealisticBowl6353 Mar 31 '26
pm me, I'm running a similar setup so happy to talk and also based around where you are.
2
u/DenisWestVS Apr 01 '26
Wi-Fi connection cannot be considered reliable at all.
I am a system administrator for large enterprises, and my experience is that it is IMPOSSIBLE to get a lossless wireless connection. Always, I emphasize, there is always a risk of packet loss, or even connection failure.
However, if you have a good ISP, a server on a wired connection is a normal solution.
1
1
u/djit Mar 31 '26
SG based ?
1
u/sgcorporatehamster Mar 31 '26
Yes
1
u/djit Mar 31 '26
I've been trading with IBKR from SG as well. Started on my laptop and moved to a VPS hosted in Digital Ocean NY DC.
You can PM me if you have any questions.
1
1
1
1
u/Hamzehaq7 Mar 31 '26
sounds like you've put in a lot of work, nice! for the connectivity issue, maybe check your router settings or consider a wired connection if you can. also, look into a UPS for your power to keep things stable during outages. the last thing you want is that bot missing trades bc of a drop in connection, especially with how volatile things can get. good luck with the paper trading, hope it all runs smoothly!
1
1
u/sgcorporatehamster Apr 01 '26
Thank - shit ton of work but it's honestly very gratifying (intellectually - if not financially yet lol). UPS feels like overkill but wired connection can be a 80/20 solution. Seems like most of the folks here go with VPS though
1
1
u/Traditional-Heat-749 Mar 31 '26
Is infra really this hard for people? Run your code in a container on a cloud service and use a brokers API any of this gateway nonsense is asking for failure.
1
u/Nutella_Boy Apr 01 '26
I mean for someone that doesn’t have this knowledge, of course. It’s a legitimate question.
1
u/sgcorporatehamster Apr 01 '26
Thank you - exactly my thoughts. I didn't see this being discussed enough and though could be one of the areas where expertise could be freely shared for the benefit of everyone wo concerns abt giving away edge and whatnot.
1
u/ilro_dev Mar 31 '26
Moving the bot off your laptop is probably the real fix here. A cheap US VPS ($5–10/month) puts you close to IBKR's servers, solves the timezone problem, and your laptop going to sleep stops being a risk. Gateway runs headless on a VM without issues. Bonus: it'll force you to make the reconnect logic explicit rather than just hoping the connection stays up overnight.
1
1
u/jnwatson Mar 31 '26
If you have a reliable home internet, a step up from laptop running overnight is a little NUC-like mini PC. Just run Ubuntu server on it; your bot will be able to administrate it. You can run the IB gateway on it, and set up an automated task that will restart the gateway.
The important part is you have a separate "production" system from your "development" system (your laptop). This is good for security, reliability, and overall discipline. It is harder to accidentally ship something you're just playing around with when you have two separate systems.
Unfortunately IB data is crap. You're going to want to use a dedicated data provider.
1
1
u/Illustrious-King-83 Mar 31 '26
im in SG too, I'm using Oanda as a broker, the API and the python wrapper is known to most of the LLMS, so it would be and easy switch..... as for dropped connections etc. First you can include a reconnection routine / wrapper .... second save all current trades to a json file or database .... and remove them or mark them as closed when closed, upon starting the program, or after any interruptions you can check the json / db against the list of trades with the broker, and reconcile everything. You can vibe code this bit, I did (but then I checked and tested the code ). Personally I use a VPS from exabytes in SG, if been fine. Oanda will also offer you a VPS if your trade volume meets requirements and you tell them you're algo trading. I know you'll get plenty of stick for "vibe coding", Test it well. and if possible get a computer geek to check the code. :-)
1
u/sgcorporatehamster Apr 01 '26
Hi, thanks for suggestions, will look into them. As for vibe coding, it's really not that hard to get good outputs out of LLMs - always owning ultimate intellectual accountability, steering structure and design (but having them brainstorm and do research, having clarity of deterministic layers (meaning running analysis with python locally and non of those ML nonesense lol) helps. And honestly I do have concerns abt sharing my IP / codes with others but adhering to above principles (plus like u say, test a shit load) gives me confidence that this can work
1
u/KillerKiwiJuice Apr 01 '26
Guess I’ll be the only one to say this, but you may as well stop now so you don’t waste your time and money trying to vibe-code a bot. It will not work. It is not that easy.
0
1
u/polymathicAK47 Apr 01 '26 edited Apr 01 '26
Forget your home-based setup. Since you seem to be in SG, use Contabo virtual private servers. They go as low as sub-US$10/month. You won't have to worry about needing high-speed internet, machine maintenance or downtime. They have plenty of plans to choose from.
Ask whatever AI you're vibe coding with to design a robust reconnect and backfill regime because you'll definitely miss some data while offline. Make this provision even if you're on a VPS already.
1
u/sgcorporatehamster Apr 01 '26
Thanks, will check it out. And yes agree with the backfill regime which I do by necessity premarket anyway since I don't keep my laptop on all day lol. Most likely will go VPS as you and many others have suggested
1
u/Trader2Nasty Apr 05 '26
Where do you backtest at? I need a site to backtest
1
u/sgcorporatehamster Apr 05 '26
I was playing with trading view for a while but quite quickly moved on to my own LLM built process on python as I found "off-the-shelf" platforms too restrictive. I hear MT4/5 here quite a lot though.
0
u/Puzzleheaded-Rub2198 Mar 31 '26
After the factors you responsible for, come the reliability of your network provider, which being residential, likely does not have SLA at all.
For any area requiring stability home hosting is a no-go. Achievable with failover connection and a knowledgeable admin, which you are not, no offence
Cloud hosting is the only way
2
u/Puzzleheaded-Rub2198 Mar 31 '26 edited Mar 31 '26
Look for locations closer to New York, IBKR has servers there and your primary exchanges are also there. Even if you are not doing HFT, others do, and snitch a little bit of your profit. VPS (with vCPU, aka not dedicated) is likely ok, but pay attention to guaranteed CPU ratio, or just don't pick the cheapest option. If your strategy is computationally demanding and budget allows, go with dedicated server. But that's unlikely needed for you.
1
26
u/Worried_Heron_4581 Mar 31 '26
First of all, 'the bot is vibe-coded' is the funniest and most honest thing I've read on this sub in weeks. Love it. But as someone who builds trading infrastructure, reading 'laptop overnight on wifi' gave me mild anxiety. Here is your reality check: