I would like start by mentioning that I’m still relatively new to the programming world and I do not understand most of the concept talked on this forum. However, I love to learn and this is why I’ve been lurking this forum for some time now. I’ve been programming for about 4 years now and it has been mostly in PHP (Planning on learning Elixir). I’ve been doing vary basic stuff with the Slim framework and Laravel’s Illuminate Database package, but now we are running into some issues at work.
We have been using PHP to run server side scripts like the following:
Updating a users database (MSSQL) by fetching users from our Active Directory. We are basically fetching all users from AD to get their username, email, phone extension, employee number and then fetching more information about their jobs from a Human Resources database. This is to create a users table for our employee web application portal.
Creating mailing list from a web interface and then sending the emails. It can be thousands of email. Any user can create a mailing list (might be up to 40000 emails in a list). These list are then placed in a queue and PHP reads that queue to then send the emails. Sending emails is actually very slow. Is it PHP or our setup… we don’t know yet.
More scripts will be implemented in the future.
These scripts are slow and/or are timing out. We’re running PHP in IIS and we’ve been getting random FastCGI problems. I feel like PHP might not be the best for these types of scripts.
I would like to know if I should be looking and learning a different server side scripting language to run these scripts. Also, what would you recommend as the queue/tasks system? Some scripts need to run as soon as possible (emails) and some scripts need to be scheduled. I do not mind learning a new language as I can see this be very beneficial in the future.
Again, I’m new to this stuff, but I’m willing to learn and would like this system to be very solid as we will have more scripts scheduled in the future.
Shouldn’t this actually be done via a trigger on the active directory to keep the secondary database in sync at all times instead of polling with a PHP script or so? Or better yet don’t copy the data and just have the other things access it from AD directly (this is what we do at work)?
Oh I most certainly would not want to custom-build this, I’d use one of the very well tested listserv servers.
PHP is actually pretty fast (especially PHP7) so honestly it’s probably not that, I’d wager it’s the algorithms used. Now doing them in Elixir would make using the better concurrent algorithms much easier than in PHP, but it is still easy enough to do the concurrent actions in PHP too.
Elixir escripts would of course be single-runnable things that you could do though. But for such simple sounding script I’m not sure it is entirely worth it, I’d probably have opted to just keep the PHP if that is what you all know but write it concurrently, or use python/elixir/ocaml, all of which have fantastic concurrent processing for such things (or even BASH itself, but you mentioned IIS, which implies Windows, which is utterly horrible, no one should ever ever ever use Windows for a Server, even my Microsoft-heavy workplace is replacing the old windows servers with linux over time).
That sounds like it makes sense, however I do not know AD. I will have a chat with the server guys.
I’m not sure how that would work. I need the username and employee number to be in our users table. So I need to copy those fields. (I’m probably not understanding something, haha). Every time a user logs in, I cross check their username and password with AD. If it’s valid, I cross check that username in our user table. If they are not in there, I create a row for them, else I update it. This table has many more fields like their department, job title, profile, etc. This has worked beautifully so far, but the mass update that we do every night has been crashing from time to times.This mass update needs to happen to make some application in our portal accurate. Like when an app works with the users department, it cannot just wait for the user to login to be updated. It needs to be updated on a regular basis. Again, I’m probably misunderstanding your suggestions and I’m sure it would be the way to go as we seem to be doing similar things. You just actually know what you’re doing and I don’t. Haha.
That seems like the smart move. I’ll take a look into this. Does did work only on a subscription basis or could we basically send email to any list of emails? i.e. Clients
Yeah, our setup is probably not the best. We will try and stick with PHP. We do have PHP7 running. However, as mentioned, it’s all on IIS.
Yeah, we’re in a Microsoft-heavy workplace. It’s a public School Board. We’ve talked about Linux to our server team and they basically said sure, if you take care of it. We are not a big enough team to start taking care of the server and programming. We will try to make Windows work, but ultimately, Linux might be a good idea in the future.
Anyway, thanks for your reply. I really appreciate it. Very helpful!
As such just use the ldap_fdw or multicorn plugin for PostgreSQL to expose the LDAP server as a table in PostgreSQL, then you can cross query all that you want while never ever going out of sync.
Yeah that sounds like duplication of data, I’d use a FDW to keep the main user information in sync then use another table to add extra information as necessary with it linked to the main table (although what you put you shouldn’t need to, AD/LDAP holds department/job title/profile/and more information too already).
There are styles for both out there, both of those are very common things to do.
Fair warning, running FastCGI over IIS incurs a speed hit on connection setup, but for long running operations that doesn’t really matter.
Heh, we are a college here!
Windows just had so many constant issues here that when they got the new server bank they decided to set it up as a VM, and since we’ve done that it is trivial to spool up more VM’s, so we set up a lot of linux stuff, they’ve found out that a lot of what they run here runs better under linux than windows, saving money on getting and running less hardware so everything has been migrating over except the AD and Exchange servers.
wow amazing idea. We actually don’t have PostgreSQL, but I found a way to do it in MSSQL. Created a Linked Server to ADSI. It’s working super well. Very glad, thank you for the idea!!
Yeah, we are so into Windows just like you guys were, however I do not think they are willing to change that anytime soon. I’ll deal with Windows for now, but maybe we could try Apache or NGINX instead of IIS.
So on the original question, just a data point that Elixir is a great language to run server side scripts on scheduled tasks - we’ve been doing that and it never missed a beat (just run a GenServer that wakes itself up every night).
But yeah, even though PHP is on my very short list of Languages That Suck So Much, They Should Be Banned (with VB6 and JS), if you think that your trouble is caused by PHP’s (perceived) slowness, it’s 99% certain that you’re doing it wrong. I’ve worked on my country’s largest site, in PHP (4, 5), and performance sucked but we were fully aware that that was entirely the fault of the 1m lines of spaghetti code we ran on our server farm (which ballooned to 450 appservers before we got to our senses and rewrote the whole damn thing). It’s pretty much always your own code, Rails, PHP, Python, etcetera have less of an impact than people think.
This, I’d recommend this, it runs circles around IIS on Windows!
Lol, I entirely agree! I may not get a high pay here but I love the people here, it’s very unicorn’ish. ^.^
In terms of raw speed, PHP runs circles around Ruby/Rails/Python, but even then it is ‘usually’ the patterns and algorithms and styles that make everything slow, not generally the language.
Now as for being productive and maintainable in a language for the long term, none of those would work (not even Elixir if not for the BEAM ^.^).
Once set up with auto-update, a linux box needs relatively little
“taking care” effort. (I have one server at day 599 since last reboot.)
Also, consider trying this out via a cloud provider like AWS, which
lets you skip the “choose” | “buy” | “install” parts of server acquisition
and go straight to “run”
I’ve never ran into such an error. Getting the list of ALL users information (about 80 thousand) takes like 4 seconds but never hit an error. I’m betting that error is from your LDAP server and not from :eldap.
Wouldn’t surprise me. As for pagination, hmm, I’m unsure, never done that yet… Even getting the entire list is super rare, I’ve only done it a few times mostly for testing. The great majority of access is a single person at a time by their main ID.
Maximum number of entries to return, and maximum time to allow search to run. These values, however, cannot override any restrictions the server places on size limit and time limit.
