I meant to give this as a lightning talk, but unfortunately the session was too short and I gave a different presentation first and it didn't come back around to me again, which is too bad -- I actually didn't care about the other presentation that much.
My premise:
Resolving Python's problems with web programming is the most important thing we can do to market Python.
You might dismiss me as being self-centered, as I'm a Python web programmer, and we all think our own problems are the most important problems. But then I'm a web programmer because I think it's the most important and empowering development environment of our time -- it has been for at least the last five years, and I'd be surprised if that didn't stay true for at least the next five years.
And I am doing okay -- these aren't my own problems I'm complaining about. I don't want to talk down Python web programming too much -- if you are serious about web programming the initial investment will pay off, since Python is a great environment. But if you aren't committed enough to invest that time, and you want to produce something useful quickly, then -- though it hurts me to say this -- Python isn't a good choice.
This is an incredible missed opportunity for Python. Most other marketing efforts underway for Python are (IMHO, of course) misplaced.
Right now Python marketing -- which is admittedly an afterthought for everyone -- is driven by a desire to compete with Java (and probably .NET recently). But people won't admit that we are utterly incapable of competing head-to-head with these languages, because they speak to an audience we can't market to directly.
Everything that makes Python great doesn't mean much to managers -- will they care about significant whitespace? Sometimes it's even contrary to what managers want. Managers are wary of empowering programmers, because it scares them to become dependent on a single programmer -- I think this isn't a big problem for Python, but it is something that encourages heavyweight methodologies (which aren't good in Python), B&D programming environments, static typing, etc.
Python appeals to programmers. That's who we have to sell ourselves to. Forget everyone else. Let the programmers sell it to their own managers -- if they really love Python, they'll go through that effort. If they are really successful with Python it won't be hard for them -- Python will sell itself. If Python can't sell itself then it doesn't deserve to be adopted -- ultimately what matters is what people build in Python, not the virtues of the language itself.
What we need to do is get our foot in the door. We need to get programmers to write projects and do so successfully. We need those projects to be visible to managers.
Web programming is the best way to get our foot in the door. A programmer with little experience can produce a useful web application in a matter of hours. Not just playful or interesting, but something that can actually go into productive and live use. The only other environment where that is possible is the command line -- and managers never see programmers' command line tools.
Web programming is also a kind of universal need. Sure, there's lots of things besides the web. But unless you really try to avoid the web, as a programmer you are likely to have occasional problems that are best solved with a web application. This is true no matter what field you are in. In part because web applications don't just touch on core needs -- e.g., embedded programming at a hardware company, numerical analysis at an engineering firm -- but on any coordination needs, and everyone needs to coordinate things. The embedded
programmer might display test results in a web page, the engineer may collect data via a web upload form. If they have successful experiences in Python in these cases, it is only natural that they will try to use Python elsewhere. That's great, because Python is appropriate in some way for most of those other use cases too.
When we consider Python marketing, we should look for accessible marketing successes. Java isn't accessible. The PSF isn't Sun, Zope 3 and PEAK aren't J2EE, and I could go on. These aren't just technical differences, they are based on deep social differences. We need to look at agile and open source successes. (And we have to look at successes, so no need to look at Smalltalk or Lisp...)
By some coincidence three languages starting with P are often grouped together: Perl, PHP, and Python. It's a nice coincidence, and these are the languages I've thought about. Both Perl and PHP have had tremendous bursts of growth because of web programming. This graph isn't entirely a representation of PHP's growth, but it says something, and it's doesn't take a graph to know that PHP has grown incredibly rapidly:
And Python could have been PHP. We could have seen that kind of growth.  But it's not all over -- PHP 5 is barely catching up to Python's features from 10 years ago. There's a lot of room for a better language to take it's place. (Though we should be a little worried that Ruby could be that language.)
Anyone who cares about Python should care about the web programming situation -- no matter what your area of expertise. Those of us in the web development community need both support and pressure to improve things. We also need support from the Python core -- there's some longstanding issues with Python that cause problems; things like long-running processes and reloading code, isolating and monitoring interpreter instances, and restricted environments. These are things that reach far down into Python the language, in such a way that the application programmers that need the features aren't generally able to make much progress (nor is it clear they'd get support on any solutions they did provide). We also need input from people trying to do commodity Python hosting, and we need to pay attention to what they say.
Really, I want this post to be seen as an optimistic one. We all know about the problem, but we should focus on the incredible opportunity that's out there if we can solve this. I know we're all volunteers -- I am too. We work on what we enjoy. But we also do this work because we can always imagine great things yet to be created. We do this work because we enjoy sharing our ambitions and successes with others. I just hope people will direct their imaginations in this direction, because we could really use the help.
I totally agree with you.
Trying to explain to people how to do Web programming in Python, or even trying to convince them to let me do Web programming in Python (instead of say PHP) has been an embarrassment. (This despite my opinion that PHP is an embarrassment to the term "programming language".)
 Once a program is placed on the Web, its functionality becomes instantly accessible to millions of people. Tiny, simple programs can become useful groupware tools. It's all about the barrier to entry.
Over the course of Python's history, i think Guido hasn't recognized the full significance of this. It's not due to any shortcoming as a language designer; it's just that the Web is not what he does every day. At PyCon i heard him remark something along the lines of, "The Web isn't everything" or "There's a lot more to programming than the Web." Well, yes and no. There's certainly a lot more ways to use a programming language than just for Web programming. But if you're talking about adoption, the Web is everything, nearly.
Improving standard library modules to support Web programming (such as cgi.py) has been on my to-do list for ages. I bear some of the burden for not acting sooner to improve things in Python, and it weighs heavily on me.
Not all is lost, though. PHP succeeded at displacing Perl as a widespread Web programming language, so maybe it can happen again. Python can still do many things that other languages can't -- for example, cgitb exploits Python's unique strengths to provide a huge win for Web developers. Huge. Using cgitb completely changed how i did CGI development overnight. Even though it was introduced a few years ago, no other language has anything like it. Things like this point to deeper strengths in the language that no amount of hacking can add to PHP.
--> Trying to explain to people how to do Web programming in Python, or even trying to convince them to let me do Web programming in Python (instead of say PHP) has been an embarrassment. (This despite my opinion that PHP is an embarrassment to the term "programming language".)
The vitrol that Python programmer's have for PHP has always baffeled me. Frankly, part of the reason I've avoided learning Python is because every Python programmer I've ever met has been a prick about my current choice in programming language. PHP works, I enjoy programming in it, I like the way it feels and reads, and the user community is incredibly supportive. More importantly, I've built some seriously effective web applications using it. If you want me to bother learning Python, loose the 'tude, dude.
Seriously, every language has its problems and it's quirks. PHP is far from perfect (e.g. no namespaces), but many of Python's language design features, such as the meaningful whitespace concept, I find unpleasant to work with. (A little too much like FORTRAN for my taste.) That doesn't mean that I don't think its a good language, or that Python programmers are bad people. (Just pooly socialized.) Obviously people have done a lot of great work in Python, and its a very useful tool. But its also a language that has a very different syntax from PHP, PERL, JavaScript, Java - languages that web people are familiar and comfortable with. If you want to get us to part with our semicolons, curly braces, and crazy bohemian whitespace, you're going to have to be nicer to us, and more polite about the tools we love.
Finding Python hosting is difficult and all are expensive. I find that a dis-incentive to try it. Even if we leave that factor out, I find the plethora of options out there confusing. I plan to try Nevow sometime in the future.
Swaroop
www.swaroopch.info
Persistent processes for servers (eg: twisted) will not be as successfull as PHP. CGI or mod_python are the only affordable way to go with Python. Nevow works partially with WSGI (WSGI knows nothing about LivePage and Guard). If you are looking for XMLHttpRequest support, try wallaby.
Deru.net is very affordable and reliable, and Python runs great there.
Now this is interesting. I am running hosting besides other things and I'd love to provide Python too. Is there a consensus what that means actually? Is it CGIs in Python, Zope, Webware, Twisted, or mod_python within Apache? Or all of them? Or anything else? Then, there is a question how to set them up when you have decided - what access rights, permissions, etc.
I have a couple of my own projects there and they use mod_python. This includes some hacking in the mod_rewrite, or mod_alias for every project that needs python for it to work. PHP is different: you just drop the files in the directory and it just works.
Maybe when ordinary administrators (= not using Python every day) know how to set it up, there will be plenty of Python hostings; when it is as easy as issuing apt-get install php4...
jbar
# Jiri Barton
This is EXACTLY why Python is losing out to PHP.
There are WAY too many options (Zope, Twisted, CherryPy, Paste, Django, Quixote, ad nausem...) and ALL of them require a long running process (no-no in most hosting scenarios) - which leaves people with CGI, which has been widely accepted as NOT the right way to build large web apps.
This means until we have totally clean Python integration with apache (mod_python is too low level for most people, we don't want to care about returning http response codes - Python is high level remember!), or tomcat for Python <- this is what I think we need most btw (and Zope is an abomination in my eyes and nowhere close to filling this need), we're suck using Python as a client side language.
# anonymous
If we have enough clear documentation for mod_python then I think it can be a killer web language. I have to read mod python tutorial 5 times to get the information that Mr.Grish states. IMO , it's a splendid tool and I am beginning to write applications in it. It all grinds down to attracting the end users with simple documentation even if it's so low level, the rest follows.
Thanks for writing this post. I think you're right on. I myself am, I suppose, an intermediate programmer -- I had only programmed in Perl and a bit of C++ before finding Python -- but I rapidly became enamored of Python's usability and "discoverability." Even after a year or so of programming Python, I still get the feeling that by just poking around in the command line, I can figure out what I need to know to get a job done without reading reams of documentation. And in fact I have managed to get things done that way -- my productivity in Python compared to Perl is an order of magnitude of difference.
Except when it comes to the web. I'm currently trying to build my first substantial web application, and I would love to write it in Python. But watching the recent progress in the Rails world has made me wonder if learning Ruby wouldn't actually be faster than building and/or learning the Python frameworks necessary to get my project off the ground. Or even figuring out which frameworks are appropriate! "There's one obvious way to do it" most definitely does not apply in the Python web programming world, and it's too bad.
Even so, I think the Subway project looks promising, and I think that Python has several features that make it better than Ruby for web programming, most important of all, I think, being that Python has the best Unicode support of any of the "P" languages. (A fact which shouldn't be underestimated.)
It seems to me that a concerted effort could be made to unify various existing components into a useable and easily installed MVC package.
The competition with Rails may turn out to be the best thing to have happened for Python web programming... I hope so.  How do we build momentum?
Agreed.
Thru the years every Python web solution I've evaluated makes at least one deadly assumption: scalability and performance don't matter, production uses compatible version control, the HTML/XML is not templatized and is not language neutral, no CGI URL will end in .html, every URL is controlled by Python code, subdirectories are class heirarchies, etc. That is why Python fails on the high end.
Python fails on the low end because PHP makes an effort to get out of the way of newbies. It is easy to get useful results out of inexperienced people without reading a lot, configuring a lot, or annoying administrators.
Solutions like Zope are closer to J2EE than to Ruby On Rails. That is a mistake.
PyGTK is a first choice. Maybe some day something like CherryPy will be too.
# anonymous
How odd. My web framework made none of those assumptions. I can't imagine every other framework you looked at made them.
Which framework? Devavu? Is Dejavu recent? I haven't looked at Python web solutions for months. A quick peek at the dejavu docs uncovers this statement.
- Dejavu uses bytecode hacks, and therefore requires CPython.
Is Dejavu still tied to CPython?
# anonymous
Dejavu is an ORM. I don't see the relevance here.# Daverz
I'm a PHP/Java/Ruby/Python programmer. I tried once to talk (but it's amazing how some people does not know how to talk) with a influent Python local dude about this. I told him that mod_python will spread more easily if it could show all it's power running on Apache 1.x, since there's is a LOT of websites still running Apache 1.x.

He did not see the point there. I was thinking in a way that could help to, for example, THOUSANDS of PHP coders try mod_python (easy), without upgrade their webservers (harder) while PHP still marked Apache 2.x as experimental. But he was not able to talk about that. I told his so beloved tool could not work perfectly (and I was able to help if there was a way, on that moment) on a older enviroment, and he became mad, offended. Stupid.
While this kind of behaviour still exists on the Python community (and believe me, there is a lot of situations like this), the tool will not even go closer where PHP is. While PHP is not a half of the language Python is (and I think Python is very better than PHP), some Python dudes are really hard to deal.
Thanks for writing this, Ian.
You've clearly articulated a deep frustration that I have felt for several years now. I would love to do web programming in Python but I know that pragmatically PHP is a better choice despite its inadequacies as a language. Finding affordable Python hosting is not easy ; choosing a framework is not easy. Even when frameworks are well documented - like modpython - I would like more than just the bare documentation i.e. books.
For commodity hosting to emerge and books to be written the community has to agree on one framework, develop it further and make it easy for new programmers to get started. I worry that by failing to recognise the importance of the web and adding further complexities to the language Python will actually begin to lose users.
This is one of the most important things haunting Python today! Thanks so much for writing this.
Recently, at work, one of my bosses asked me what I would choose to write a potential web application and web service. We develop a lot of Python applications, and its definitely my favorite language. After a lot of thinking, I responded "Rails." I don't even like Ruby very much, but Rails is obviously the best web framework out there because of the tightly integrated simplicity of it! Rails is the Apple of web frameworks -- one tightly controlled stack. Sure, you may lose some flexibility, but you gain beauty, simplicity, and ease of use! I really really wanted to pick Python, but I just couldn't do it. There are too many frameworks, none of which are as good as Rails.
Python is making the mistake of copying Java when it comes to web frameworks: choice is better. We are going to end of with a myriad of tools that do one thing sorta well (for Java: struts, hibernate, tapestry, et al... for Python: Cheetah, Nevow, Quixote, CherryPy, et al), but when you put them together, you get highly abstracted, loosely coupled, confusing mess. I am especially afraid that we are going this route because of the WSGI. Don't get me wrong, its a great idea, but its not the solution to our problem: its an enabler. "Now you can have more choice" ... thats not appealing. Thats more work. Thats more to figure out. In the Ruby world, if you are developing a web application, there is no choice: there is Rails. And thats okay, since its so damn well done. In the Python world, if a web framework doesn't do something that you want, you just create your own framework... why not fix the original one? The whole situation is embarrassing.
I don't think Java is the right platform to emulate when it comes to web application (ever, really). I think, at least in this one regard, we need to look at the principles behind what makes Ruby on Rails so successful, and come up with something to counter it in Python. Otherwise, I promise you, Rails will be the dominate open source, dynamic web framework for the next few years at least.
Anyway, sorry to rant here, but your post really struck a chord with me!
I don't disagree with Ian's diagnosis of the problem (although it's not terribly relevant to me personally), but I will disagree with this particular prescription for it. I can understand being frustrated with the proliferation of web frameworks, but let's face it: there is no way you are going to be able to convince anyone to give up on their pet framework to work on someone else's.
I see a lot of people in the Python web community complaining about proliferation of frameworks. I don't see anyone at all standing up and saying "My web framework X is unnecessary, I will abandon it and spend my web-framework development time working on web framework Y instead."
Let's focus on what is going to make web frameworks seamless to use. Personally I think that "web programming" is a red herring and the problems with Python are basic platform deployment issues: for example, the lack of a packaging system makes the simplest task in web programming - downloading and setting up a framework - unnecessarily difficult.
For those of you that are going to heed Ian's call to arms, please consider productive suggestions that people can follow. If we're not each going to give up our individual toolkits, let's at least agree on the featureset that would make each of them really attractive to new people coming to the Python web space.
For example, Alex Levy suggests that the problem is documentation: http://mesozoic.geecs.org/cogito/archives/000152.html
Well, I personally have resisted the urge to develop a new framework (an urge I've felt many times over the years), and I actually am willing to give up my framework of choice for more conformity. But I can't abandon my framework (and I can't expect anyone else to do so either) -- other people have trusted my opinion and choice over the years, and it would be irresponsible of me to abandon them and those applications I've developed. Which is a bind a lot of us have probably been in. That's one of the things I see in WSGI, is a path out -- a way to support the old apps and consolidate and improve the new apps, without being bogged down in abandoned frameworks and out of date setups.
Talking to people at PyCon, I don't think I'm altogether unusual. First, there's a lot of people with homegrown frameworks, and from what I can tell this describes most of them -- they are dissatisfied but trapped by their own legacy. But even beyond that there's hope that open source frameworks can consolidate. If we can phrase one framework in terms of another there's the opportunity for the lesser frameworks to melt away, to create a layered environment instead of competing full-featured stacks (well, they are all actually partially-featured stacks).
As to Rails: I agree that it's a challenge and a competitor, but it's not a model. People have been working on Rails-like things in Python for a long time, with various successes and failures. But Python's flaw isn't that we don't have any interesting frameworks or interesting MVC systems. Python simply isn't at the same place Ruby is. We have to fix our specific problems, not immitate someone else.
Okay, first to respond to Glyph's statements:
"""I don't disagree with Ian's diagnosis of the problem (although it's not terribly relevant to me personally), but I will disagree with this particular prescription for it. I can understand being frustrated with the proliferation of web frameworks, but let's face it: there is no way you are going to be able to convince anyone to give up on their pet framework to work on someone else's."""
I am not saying that anyone has to give up their old software. And Ian's suggestion for the WSGI as a "way out" is a good one. However, I do think its reasonable to centralize around one solution. I am not saying that there can't be other solutions, I am saying that our current solutions suck. If any of our solutions were as good as Rails, they would have a dominant user base in the Python community. I really believe that! I don't even care if we start from an existing base, as long as we get somewhere!
"""I see a lot of people in the Python web community complaining about proliferation of frameworks. I don't see anyone at all standing up and saying "My web framework X is unnecessary, I will abandon it and spend my web-framework development time working on web framework Y instead.""""
Well, I don't have a framework that I have written that is out in the world. However, if I did, I would be willing to give it up or allow it to become the new central framework. I think its really selfish to hold onto one's own framework if presented with the option to unify. Let's be honest, there are probably only 3 or 4 different Python web frameworks with any traction whatsoever. This adds up to somewhere between 5-10 people (likely) that actually have control of these frameworks. I see no reason why 5-10 people can't come together and work as a group to improve the obviously broken situation we are in now. What we currently have is good for noone: especially the Python community.
"""Let's focus on what is going to make web frameworks seamless to use. Personally I think that "web programming" is a red herring and the problems with Python are basic platform deployment issues: for example, the lack of a packaging system makes the simplest task in web programming - downloading and setting up a framework - unnecessarily difficult."""
Wow. I really disagree. This might make us slightly more popular, but it certainly wouldn't make us better. You are copying Java. We need Servlets, JARs, and a few specifications, and we will be good! Ever developed Java web applications? It sucks compared to Rails. You are using the wrong benchmark.
"""For those of you that are going to heed Ian's call to arms, please consider productive suggestions that people can follow. If we're not each going to give up our individual toolkits, let's at least agree on the featureset that would make each of them really attractive to new people coming to the Python web space."""
Why wouldn't we give up our individual toolkits? You can't win if you aren't willing to change. You are saying the best we can do is stagnate. I totally disagree.
"""For example, Alex Levy suggests that the problem is documentation: http://mesozoic.geecs.org/cogito/archives/000152.html"""
Insufficient solutions with good documentation are still insufficient. Show me something that is as unified, cohesive, and useful as Rails, and is just missing documentation. It doesn't exist.
Now, for Ian's response:
"""Well, I personally have resisted the urge to develop a new framework (an urge I've felt many times over the years), and I actually am willing to give up my framework of choice for more conformity. But I can't abandon my framework (and I can't expect anyone else to do so either) -- other people have trusted my opinion and choice over the years, and it would be irresponsible of me to abandon them and those applications I've developed. Which is a bind a lot of us have probably been in. That's one of the things I see in WSGI, is a path out -- a way to support the old apps and consolidate and improve the new apps, without being bogged down in abandoned frameworks and out of date setups."""
I agree with you here Ian. Lets make the WSGI just that -- a way out. But, we shouldn't try to make it the Servlet's of the Python world!
"""Talking to people at PyCon, I don't think I'm altogether unusual. First, there's a lot of people with homegrown frameworks, and from what I can tell this describes most of them -- they are dissatisfied but trapped by their own legacy. But even beyond that there's hope that open source frameworks can consolidate. If we can phrase one framework in terms of another there's the opportunity for the lesser frameworks to melt away, to create a layered environment instead of competing full-featured stacks (well, they are all actually partially-featured stacks)."""
Thats very true, and I think you are on to something here. Please, keep preaching it. Maybe you could organize some "summit of web frameworks" or something :) Get the authors of the disparate solutions together, and come up with something better.
"""As to Rails: I agree that it's a challenge and a competitor, but it's not a model. People have been working on Rails-like things in Python for a long time, with various successes and failures. But Python's flaw isn't that we don't have any interesting frameworks or interesting MVC systems. Python simply isn't at the same place Ruby is. We have to fix our specific problems, not immitate someone else."""
I don't think we need to clone Rails in Python. Rails is a very ruby-like solution, and there are certainly things I dislike about it. I am not saying we need to model it by copying it: I am saying that we need to model its principles. Its a VERY RUBY, tightly integrated, complete, simple, and fully-functional framework for building web applications from the database all the way up to the top. We need to create a HIGHLY PYTHONIC, tightly integrated, complete, simple, and fully-functional framework as well. It doesn't have to look anything like Rails, but that is our competition. That is our target.
I am not arguing that we need to create an imitator: but we do have to compete on the same level of simplicity and ease of use. PHP and Perl were wildly successful web application languages for some of the same reasons as Rails: simple to install, simple to understand, and documented copiously.
Why do people aim so low?
"I am not saying that there can't be other solutions, I am saying that our current solutions suck."
What? Zope sucks? Quixote sucks? I'm sure the developers and communities of those and other solutions would certainly feel encouraged to hear that kind of judgement fall on their hard work (and success, too).
"If any of our solutions were as good as Rails, they would have a dominant user base in the Python community. I really believe that!"
Well, aside from the fact that any incursion into the Rails site to see what the fuss is about tends to set off my anti-hype alarms (even more so than the JBoss site, circa 2001), and even when I've hit the "master alarm" I just get presented with various JSP-like endeavours which are, after the "scaffolding" is set up (to borrow the Rails terminology), arguably mostly as easy in a JSP environment (albeit without the transparent SQL-based persistence supposedly provided in Rails, and yes, a "to do" list tutorial isn't going to show off the power of Rails, even if the only Rails applications seem to be "to do" list managers, anyway), I must regard the framework domination claim somewhat skeptically.
Ages ago (ie. 1997) solutions like Bobo provided a clearly better means of developing Web applications than just using cgi.py. What then happened was that Bobo grew into being Zope, and that many developers did indeed follow that path, thus forming the Zope and Plone communities which seem in many respects to be disjoint from what is now considered to be the mainstream Python community - in a way, a dominant solution did emerge, but only as a parallel community. Meanwhile, in the wider Python community and inspired by various other Web programming paradigms, a number of frameworks emerged and matured to produce the ecosystem that you see today. Certainly, Quixote has a certain number of advocates, but probably nothing like the number of the solution that is said to have inspired it: Zope.
"I think its really selfish to hold onto one's own framework if presented with the option to unify."
 Is it selfish for someone not to abandon a framework because it is used in a production environment and that such abandonment would force them to migrate their existing applications for free just for those applications to function more or less as they did before? Sure, you get eventual benefits from your application suddenly using code that someone else is maintaining, but then there's always the temptation to open source your own framework and to try and get the maintenance advantages of a larger development community whilst having everyone else migrating their applications instead. (I suppose that this might explain why so many competing frameworks emerge, even though they often resemble one another - it's just the "selfishness" of not holding onto one's own framework...)
"Lets make the WSGI just that -- a way out. But, we shouldn't try to make it the Servlet's of the Python world!"
Well, WSGI doesn't really address anything other than a very basic standardised environment; not that there isn't merit in addressing that area, but I was disappointed that people were happy with just settling for that, and somewhat disturbed that for many people WSGI is just something to believe in whose existence acts as an excuse for not getting to grips with the harder issues.
Personally, I don't care about the WSGI level - I'd rather examine the higher-level concepts which applications use directly to behave in the ways that they do. After using Webware a fair amount, I wanted to experiment with other frameworks if only to avoid various pitfalls and issues that Webware had, but I didn't want to rewrite my framework and applications to make them run on Twisted (for example). In the end, I created WebStack to enable this whilst papering over some of the bizarre behaviour of various frameworks. For me, WebStack "scratches my itch" because I can now ignore framework issues and concentrate on actual applications.
So for me the framework wars are over. I don't care about the "distinctive" things that many frameworks promote (ie. the special template language or the URL dispatching scheme that should in any case really be determined by the application being written) - I just want a sane API which mostly returns request parameters, streams, sessions, cookies and so on and which doesn't force me to have to reinvent several wheels in every application I write, just because of some misguided minimalist agenda on the part of the framework designer. And if I want my application to appear at http://mysite/zope/folder/myapp even after prototyping it on BaseHTTPServer, then so much the better!
I agree with you in part Paul. And I shouldn't have used the word "suck" to describe any of the existing frameworks, as they still beat the pants off of anything available for PHP or Perl. The people who work on those frameworks can and should be proud of them. But, I do believe that they don't stack up to Rails. I agree that their hype machine is running at full steam, and that is pretty obnoxious. But, its hard to deny the traction they are getting.
And, no, they aren't doing anything all that impressive technically. People have been writing bits and pieces of the Rails framework in Python for years and years. But, they aren't putting all the pieces together in a simple way and working together to create a singular foundation. Rails is doing it well, doing it all, documenting it, and getting it publicized. I just wish that we had something comparable.
Personally, I don't care about the WSGI level - I'd rather examine the higher-level concepts which applications use directly to behave in the ways that they do.
That's what my talk at PyCon was about, and I want to re-express all those same ideas here as articles. WSGIKit is an architecture that uses WSGI as the way to pull together those higher-level concepts -- it's using WSGI as much more than a generic way to connect to a server. If all WSGI did was connect apps to server, it would be useful but indeed quite boring -- but WSGI is really a sneaky way of creating a standard request and response object, and a pattern for providing a stack of filters. And that is a big deal, even if it isn't an end in itself.
> What, ZOPE sucks?
No, wouldn't say so. However:
I'm a python enthusiast, but up to now, I wasn't able to find ANY python web development framework comparable to J2EE with respect to the learning curve.
I took some time and tried several python solutions:
CGIHTTPServer and low-level from the standard modules It is fine as a sample and easy to learn, but far from being production based. It took me days to find out how to write a multi-threaded server. I could help very much if there wasn't only the 5-line sample code, but also a multi-threaded server program that at least works fast enough for in-house use.
And, even worse, though the HTTPServer and cgtib are both standard modules, there seems to be no easy way or example of how to actually let them work together.
What we need is at least a basic standard solution using a multi-threaded CGI server together with the cgitb routines.
twisted Fine ideas, very promising, but needs a complete new way of development. I read the docco several hours. It is just too different from everything else, so it is ruled out. You'll never set up some hello world examples after 2 hours like with JSP.
mod_python The best standard solution, I think, but: I agree with some others here. The main problem with mod_python is the lack of a good Apache 1.3 version. For example, we are using Oracle Application Servers for production (based on Apache 1.3), so mod_python is out of the race automatically.
ZOPE Zope is a world of it's own. Just like twisted, there's too much to learn for a quick start. And with all those different template languages, it's confusing.
In my work, I'm forced to use J2EE. I don't really like it, because * I have to use Java (better than C, but ways behind Python) * Development is SOOOO slow * J2EE is a very complex world now.
But: it is very easy to start with. Every java programmer can write a Hello World JSP and test it within a few hours. The difficulties only follow later, as soon as things like session handling, DB connections, error handling, reliability etc are considered. But at that time, the devloper is already convinced that - in principle - he can build a solution with J2EE.
A python solution should make it easy to start learning from scratch, that's the pythonic way to go! (just like JSP/J2EE does...). And it should be in the standard module library.
# Henning von Bargen
"twisted Fine ideas, very promising, but needs a complete new way of development. I read the docco several hours. It is just too different from everything else, so it is ruled out. You'll never set up some hello world examples after 2 hours like with JSP."
The web is only a small part of twisted. Or did you mean Nevow? I think you could pick of the basics of Nevow very quickly without knowing much about twisted, though you do eventually need to pick up some twisted basics like deferreds, adapters, realms, etc. to be fully proficient with it. The main problem with Nevow as an application framework is the lack of integration with an ORM and the immaturity of some parts (e.g. formless needs a lot of work to be useful for more than simple forms.)
HMMM...... I guess that's why everyone uses PHP instead of Python: "..because it's superior to PHP." It's hard to believe, as a Python/PHP/Java/.NET/C programmer, that Python is "superior" to PHP.# James Rickmond
John, Rails is far from the only web app framework for Ruby. Other competent ones (i.e. used in production) are Nitro and IOWA. Several others exist which are probably little more than hobbies. That's not to put them down: some of them showcase excellent techniques and are an important part of the ecosystem.
Anyway, just wanted to clear up that misconception. Rails is, IMO, the best webapp framework for Ruby in every way, and it's certainly the dominant one, but it has prospered _despite_ being one of many choices. It wasn't a Ruby community effort to present a dominant framework to the world; it was the hard work of one person.
The "too much choice" argument regarding Python web programming has a valid point, but it's not fully convincing.
# Gavin Sinclair
This is very true. In my environment (database support, FWIW) I find that most of the utility-type programs I write fall into one of two categories - server-side utilities/daemons (with a command line interface at best) or web applications.
For web applications, I'd love to use Python. I've tried, often enough. But this is "internal" software, and it's hard to get any time to spend on it. So I have to be able to get a basic, good looking (I know, but it matters to management :-() application, up and running fast. Python can't really do this, as things stand.
The type of application I'm talking about isn't fancy - a very basic CRUD (Create, Report, Update, Delete) application is all I need as a start. Once I've got that sanctioned and running, adding the extra bells and whistles is a lot easier to get agreement to. This is where, in my view, Rails scores - it's a "build the basic application in a day" environment, with the flexibility to grow once the initial system is in place.
FWIW, being an Oracle environment, we've started using Oracle's proprietary HTML DB system. It's nice for that basic CRUD system, and it's got a reasonably nice-looking set of default templates (with slick bells and whistles like tabbed dialogs, which always impress people :-)). But it's an unholy nightmare to extend. And yet that doesn't matter, because the initial development is so simple...
So there's a challenge. If someone were to develop a Python framework which made writing a basic CRUD application a simple matter, coupled with some good, "professional-looking" templates, I'd bite your hand off to get it.
Paul.
Paul: We've had the exact Python Web framework you're asking about for about a year -- on-the-fly ORM generation, dynamically-created admin interfaces, stupidly-simple CRUD and much, much more... It's just a matter of convincing our higher-ups to open-source it. We're working on that.
Stay tuned. And if you (or anybody else) have suggestions on how to convince management to open-source a product, please contact me. (holovaty.com/contact)
I'd like to pipe in with my agreement regarding the true problem here. SQLObject is a good, transparent ORM. Nevow, Quixote and CherryPy are all quite reasonable object publishers. Cheetah or one of the XML template systems (ZPT, Kid, etc.) are decent template systems. The parts are all there. Because of distutils, they're even pretty easy to install.
The problem is that once you've got them installed, you're sitting there with a blank directory, wondering what to do next. Reading through the docs of the various projects will give you some ideas, but you don't get a great feeling about things because you're not really sure how to kick things off and make something visible that you can put your arms around and start improving. I do test-driven development and find that it helps, but a project really takes off once you've got something to grab onto.
That's what Rails offers that none of the current Python solutions does: a way to get going nearly instantly. So much so, that they can create a 10 minute demo video.
A large number of applications are all about CRUD. Sure, they all have their own wrinkles, but if you can give someone a quick start with CRUD and easy ways to grow from there, you've got yourself a winner with a big chunk of apps.
Subway can do this (but, I haven't looked closely enough at it to know if that's where it's heading). Bringing this kind of CRUD quickstart would at least let Python catch up with Rails.
As Ian pointed out in another posting, and you say here, Python has all the bits. What it lacks is the "quickstart" appeal. Whatever you call it, that's what's missing. And I don't see this as "playing catchup to Rails" - it's a simple case of providing entry-level, tutorial type documentation and sample material (something that open source projects are notoriously lacking in, because none of us like writing it).
I don't think there's any need to evangelise a "one true system" for object publishing, or for templating, or whatever. Each has its strengths and weaknesses. But picking one set (and accepting the compromises) and doing a really good job of putting that quickstart tutorial together will add something extra to the mix. Subway sounds interesting, but a first glance makes it look like (more) packaging and "layers". I'm happy with a toolkit of "bits". I don't really want a packaged solution, I want a first step up.
Heck, I'm very close to going it alone, and building something for myself. I found a document the other day called "Four Days on Rails", which went the step past the 10-minute demo video, and showed how to start customising the initial application. I'm seriously considering doing something similar for Python, starting from SQLObject, Cheetah, and Quixote (or maybe CherryPy). But my graphic design skills are dire. I'm going to find it very hard to write something along the lines of "look - it's easy to build a basic web application in Python" when I can't even stomach the look of the result myself :-)
I don't know if I'll ever get round to doing this (my life is a mess of part-completed projects...) but I'm not sure that waiting and hoping someone else will do it for me is going to work, either.
(And I even looked at Rails, but I can't get my head round Ruby so I'm probably safe from the temptation to swap languages for a while yet...)
Seems like your dreams have come true: http://subway.python-hosting.com/
Jonathan, I'm going to agree with most of your statements. But I'd like to tell you a little something, because you might find it enlightening.
You said, "Everything that makes Python great doesn't mean much to managers -- will they care about significant whitespace?"
Recently, I did a risk analysis for Lockheed Martin where we compared some of the major up-and-coming popular "scripting" languages. I chose Python, Ruby, Lua, Ocaml, and Erlang as examples of languages with strong but somewhat hidden support. These are the folks that I feel are the up-and-comers. So we did an analysis, comparing stylistic differences, typing, testability, readability, usability (as in number of libraries), and maintainability (what companies could offer sustainment with these languages). The risk scale went from white (0 risk) to green (not too risky) to yellow (risky) to red (too risky).
Python was the only contender in this list that's in the green. When I spoke with middle-line managers and the hollow once-engineers who hadn't done any real work for years--now doing "engineering management" and "proposals"--Python had the most favorable impression. They loved the idea of things like significant whitespace, because (and I quote this), "It makes people's code more similar." It also helps reduce the burden of people who draft up code standards. Other popular features were garbage collection, familiar syntax, and the fewest "new methodologies" for programmers to learn. Now, maybe my bias is showing here, I'm not a huge Python fan. For that reason, I had the other engineer on my project, who is a big Python fan, handle Python. I thought it was interesting that the same things I thought were weaknesses of Python turned out to be strengths in the eyes of our review.
But as I was doing this comparison, I realized something. If the Revolution were to happen today and Java and C++ were shot in front of the wall, Python would be one to rise to dominance. Because, when you get right down to it, Python is the least different.
# pdx
Ah sadly one of the great recurring themes in the python world.
I (and others) have repeatedly had this discussion. As a non uber-geek (in python or elsewhere) I can say the impression I've gotten over time, repeatedly, is that many people who write python are too damn smart. They say "look, we've got great libs that do X Y and Z. Just put 'em together!" Fine. If you're a really good developer, that's likely workable for you. Especially if you're working on a large complicated project, with lots of other really smart people like yourself.
But there is a huge class of ("under")developers, who either don't have the skills, experience and/or time to do this. It is these people who end up falling back on PHP, or increasingly, Rails, as it caters to a more RAD approach than putting together python libs. And the sheer numbers of these people have a huge effect on public perception and language penetration - that PHP trendline surely isn't measuring high-quality applications. Yet decision-makers see that trend, and are reassured; publishers see it and see "okay, we can publish a book on this"; ISPs see it and say "okay, we need to look into supporting this". Python developers seem to see it and say "but, yeah, almost all of those apps are crappy and written by script kiddies". Yes. Probably, but that's not the salient point here IMHO.
For example, Quixote is very nice inside - but it is certainly not packaged(docs/site/etc) or presented as something Joe Developer can start running with. It is clearly used and aimed at experienced and smart people. That limits its exposure. Which is probably fine - less support needed on a mailing list, and they have a great tool for their needs. Zope/Plone is just not something many will consider, for various reasons. It's complexity and opaqueness being the initial ones, but that's a near-religious argument I'm not looking to get into. CherryPy, again nice, but pretty barebones still last I checked. Great if you want a base tool to build on. Maybe akin to a java servlet container, but not a cohesive stack.
Also, a key point on Rails that David (original coder, current leader) mentions often, is that the framework was extracted, not designed on paper. And, less mentioned, but to me seemingly as important, it was basically done by one person. The original 'spirit' is very cohesive because of this - like a novel written by one person compared to being assembled and written by committee. This is why I'd guess projects like Subway will have trouble "gelling" into any similar sort of cohesiveness. Not that it's impossible to blend frameworks seamlessly, but I'd say much harder.
I only hang around these discussions because I would much rather write python than ruby, but Rails has presented an unbeatable argument at this point, that nothing in python matches. And I'm far from the only one with this opinion. I think the only way a viable competitor will arise is if a small team or individual takes the initiative and does it - summits, merging projects, etc. are not likely to work based on past examples. Perhaps the project/framework mentioned by holovaty.com (Adam? sorry can't see the comment while typing this) will be one to take this shape, arising out of a small team I believe (kansas.com?).
Okay enough from my peanut gallery. Thanks for raising the issue again - hopefully for the near-last time?
I think you are right on the money! This is analagous to the ballyhooing that comes from the Postgres group about MySQL. Sure, it's got the bells and whistles and can also make sandwiches, but it was lost on them that it was the ease of use coming in and relaxed learning curve of MySQL (not to mention that it could also run on Windows) that helped it's popularity.
I have this sinking feeling that new easy to grok simple languages will allways spread faster than OOL's. It's easier for a noob to understand logic first then paradigm later. If he doesn't have to deal with learning OO (as an example) at the onset he gets down to the business of writing code a lot faster than one who does.
Seriously, the group of people that you want are the young kids that some have termed "script kiddies". They are the ones that get excited about it and turn it into something cool. As far back as '98 people were talking about how cool PHP is and before that they talked abuot how cool Perl is. But nobody was talking about how cool Python is. That talked seemed limited to people on high horses looking down on the hordes of kiddies. A gret many of whom grew up (a little at least) and began influencing those (like managers and publishers) around them. And if that influence wasn't at first by jaw, then it was by example.
Please don't take this the wrong way. I am someone that makes a living with PHP, but I just adore Python (inspite of the fact that I also love the Curly Brace). It's an awesome language! However, if it's a popularity game we are talking about, simplicity is the ulitmate trump card. PHP has that.
# BDKR
I think you're making what I call the "Microsoft Argument": target the mean of the standard distribution of brainpower, not the starboard tail, where the pythonistas hang out. This may be an increasingly bad problem with python. I dimly grasp what we're doing with decorators, and really haven't gotten my mind around why a 'new style class' is better than an old style one, and the use case for meta-classes escapes me. Maybe the answer, then, is a custom apache distribution that automagically configures mod_python for intense http combat. Not that I've the technical chops to configure such, mind you.
# Chris Smith
Re Ruby on Rails - personally tried enough frameworks to want to wait a long time before committing real effort to it. Have a rule these days not to take frameworks seriously until someone is able to describe how it "sucks". At that point the sort of insight required to build real apps with it starts popping out. ROR is still on the hype curve right now.
It's great that it's become a focal point for many developers, something lacking for any equivalent Perl, PHP or Python framework. That guarantees a longish future and no doubt plenty of supporting libraries, both important for betting your development project on it.
But The make-or-break for me is the "edge cases" (the moment you step beyond CRUD), plus how things start to look when the feature list grows.
Specifically, a couple of issues I have from doing nothing more than trawling ROR documentation (which should probably be directed at a mailing list but David H scares me;), so here goes...
I've seen the approach of mapping actions directly to methods before in a PHP framework called ISMO (http://ismo.morrdusk.net/). From trying one project in anger with it - started off nicely but breaks down when you start needing to implement things like a authentication with permissions global behaviour (read Intercepting Filter) which needs to be bubble through to individual actions. Also things like an Application Controller, such as for a form with multiple parts, required blood. Perhaps Ruby can prevent the same happening with Rails in a way that PHP can't but need to see that before I believe.
Otherwise Ruby on Rails lacks ease of deployment right now. As I see it only "ASP" type technologies can deliver that, number 1 now being PHP. And ease of deployment is not just a feature for newbies. It opens up options like "just in time" code generation and caching which, in turn, create new possibilities for optimization and scaling smoothly. Along those lines, another rule I have these days is if it does l10n at runtime, move swiftly on.
Sick as it may seem, I'd recommend exploring Python as an active generator for PHP (i.e. you never touch / extend the PHP yourself); Python working as the design / modeling tool. In general PHP "works best" when it violates all rules of sane programming (other than security of course); as raw spagetti, perhaps because PHP was always meant to be just a template language.
The spagetti nature of PHP can be preserved while employing sane coding practices by using code generation. Right now the tools aren't there and no one's really explored the potential. The furthest walk I've seen in that direction is Scriptol - http://www.scriptol.com/ and some template engines written in PHP itself, such as WACT's http://wact.sourceforge.net/index.php/TemplateComponentArchitecture.
Stepping out of what platform makes most sense for a moment, PHP's installed base represents opportunity for anyone writing applications which target it and figure that's the bottom line. There's a joke, where I came from, about the answer you'll get if you ask a local farmer for directions: "Ooooh, you don't want to start from here!"
Sick as it may seem, I'd recommend exploring Python as an active generator for PHP
Yes Harry, that does seem sick. Sick! Horrible! The debugging horror! The deployment nightmare! That is the framework I really, really don't want to see... maybe if you keep it in the basement and never tell the neighbors it was ever born it would be okay. But do not reveal such a thing to the light of day, please!
Oh well, guess you don't want to see this http://eric_rollins.home.mindspring.com/pgen/ - Ruby again ;)
Apologies - got rant some more;
"Sick! Horrible! The debugging horror! The deployment nightmare! That is the framework I really, really don't want to see..."
Perhaps as a general framework, yes but for more specific application, where the design stage involves a narrower range of choices, it can work well.
There's a perfect example I forgot - Worksheet Server: http://www.jedox.com/. Design / development is done with Excel and the generated PHP code is not meant to be touched directly - it behaves something like a "component" on the web server. Further changes are made in Excel, overwriting the previous set of generated PHP code. So PHP acts as something like the "byte code" of a Worksheet Server "runtime". Did a review of it a while back http://www.sitepoint.com/article/php-apps-excel-worksheet-server.
And a potential candidate would be tools like JAlbum (http://jalbum.net/) which normally spit out HTML. Could easily be used to spit out PHP and add all those features some people seem to appreciate on their galleries, such as hit counts and comments.
Otherwise the devil is in the details - I don't see debugging horrors and deployment nightmares as givens - depends what you're doing and how you do it. Sure anything that involves generating code will itself require significant development overhead but
As one simple example of Python aiding PHP development, using Ned Bachelors COG, you might have something like:
<?php /* [[[ import cog from myapputils import getdsn dsn = getdsn() cog.outl("$db = DB::connect(%s);" %dsn) ]]] */ # While developing use this (will be replaced by COG) $db = DB::connect("pgsql://devuser:password@localhost/myapp"); //[[[end]]] # Continue with PHP script... ?>The PHP script is still executable while hacking / debugging is in progress. Deployment could be a Python script that runs COG over the source then copies it to the producive environment, replacing previous versions.
Anyway...
I think it's reasonable when using static publishing to produce PHP. I'm not against all PHP generation. But I think it's important to keep the generation dumb; set some variables, include some other PHP files (which are custom written), and that's about it. It's the modeling in Python and generating PHP that would scare me.
I've been shopping web frameworks for 3.5 years now. Really just window shopping - I've had project ideas but not the inclination to pursue them in my limited time. Now that I've finished school and only work full-time this has changed a bit, and I've gotten more serious about a project. I still don't know if it would have gotten off the ground except for Rails.
I've developed with MS technologies for about 7 years and the last couple has been mostly web apps with ASP.NET/C#. For a long time, I've been deciding which Java frameworks I would use for my independent work...I've played with all the struts, spring, webwork, tapestry as well as a couple ORMs (thought I really liked Hibernate)and had pieced together what my "stack" would be. I really like the toolsets and active community there. For a long time, I've also been in love with Python but have never really gotten past "Whetting your Appetite". CherryPy really peaked my interest at one point and I even found well-priced hosting that supported it but I never went anywhere with it. I remember playing with Zope (at least, Zope DB) a little bit but didn't get far. Cannot say why now (it was quite a long time ago) and unfortunately I'm out of the market for the time being.
Maybe I am just a sucker for hype but Rails got me moving. I found a hosting provider to support me (and for a bargain rate, great service so far). Once I was sure I could actually deploy my application I started building it. This is after doing nothing but idly looking at various frameworks, reading various websites, following several communities over a period of years with only a few concepts for potential projects in the back of my mind. Rails HOOKED me. I had barely learned anything about Ruby before learning about Rails (what little I had learnedI liked less than Python, and as it didn't have a web framework of note I didn't pursue it further). I think Rails has taught me somethings about Ruby that I now appreciate more, but I still have a long way to go and ultimately don't know how I will feel about Ruby as a language.
Please understand, the rails tutorials and scaffolding have done nearly as much harm as they have good for Rails (and that is saying a lot). Rails != scaffold.
Yes, Rails views look like ASP/JSP/PHP spaghetti code. They are different, and you can find out why if you read more of David's holy writ but even better if you experience it for yourself. The framework guides you by convention to do things properly. Coming from standard ASP & ASP.NET, I never fully grokked MVC (I understand it, but I haven't lived it). I was often paralyzed by questions of "where does this type logic go". Rails seems to be telling me, and telling me in ways that I don't feel anxious or guilty about, and telling me quickly.
Really what attracted me to Rails though is not the hype or the scaffold or the 10 minute demo, although that certainly helps when I feel I'm making really rapid progress. What attracted me is the philosophy. Don't repeat yourself. Use convention instead of configuration. After using ActiveRecord going back and doing Hibernate (even with everything its tools can generate for me) really makes me sick. After using ActiveController and ActiveView, the idea of the struts or springs XML config approach makes me sick.
Truly, I don't have the Python experience to speak to it in ways I can speak to Java and MS technologies. But Rails sold me in minutes, and I've been shopping a long time. If the Python community can develop something as cohesive and/or market it as well, then it can win. Python I think is unquestionably a better general-purpose language than Ruby still, just based on its library and tool support as well as it its distribution and mind share. But you might not have that much time.
# JeremyH
I totally agree with your thoughts. However, I think the solution will be simpler than you think: Through Ironpython, Python will be able to leverage all the power of ASP.NET running on the .NET framework and Mono.
What can be better than that?
# Luis
.NET is very, very similar to Java (the platform, not the language). We're talking about wanting very high-level web frameworks, and I know that the closest you get on Java is RIFE. I don't know about .NET.
The trouble is that frameworks for .NET and the JVM are designed primarily for statically typed languages that don't offer cool features like metaclasses. I've used Hibernate extensively, and I can safely say that SQLObject (and ActiveRecord, I'm sure) are much easier, because the language offers features to support that kind of thing.
So, while it may be nice to be able to call .NET and Java code, that is not really a big win in terms of web development productivity.
I think you forgot to define "Web Programming".
It can be anything from help with making HTML to Content Management Systems to Database Accessing Frameworks to advanced templating and forms handling.
Depending on the audience, different solutions are needed, and I can't tell what audience you are aiming for. This is too hand-wavy a problem definition to yield any useful answers.
# Jacob Hallén
I think the problem is one of "product design" and "usability analysis" more than anything else: What's lacking is a single easy to pick up choice that combines the parts that already exist, documents them well, adds easy tutorials, and makes it easy to package and deploy the resulting web apps.  The current choices for Python web development seem either in involve way too steep of a learning curve or aren't really complete on their own.
This is said w/o deep knowledge of most of the frameworks out there, just my general sense from talking to people and dabbling a bit with a few.
# anonymous
I haven't looked at Rails at all, so I don't know what we're up against, ;) but it seems to me that something that support representational state transfer (REST) really well could be very useful.
I haven't found anything that does...
# Magnus Lyckå (Who dislikes systems that assumes that ASCII is enough)
Can you guys get down in writing just what is so good about Rails that you would like to copy in a Python framework? There seems to be a lot of "it's so great, I wish we could do that", but how about a top ten list of what's so great. E.g. "easy to install", "make CRUD screen from SQL table definition" etc.# Codeless
<plug> I'm yet another developer who set out to create the nicely integrated tight-stack RAD web app framework for Python. However I didn't start from scratch but instead chose Quixote as the underlying framework (very Pythonic, if not the most IMO) and built up from there. The result of that is http://www.qlime.org/. Anyway it was interesting going through the article and comments to see what people are looking for. I'd be thrilled to get more feedback regarding my framework from end users. QLime hasn't reached 1.0 yet and significant design decisions can still be made. I'd rather be building what people are going to use. </plug>
One of the reasons why there are many frameworks is that Python is inherently easy and inviting for developers. Why learn the details of an existing framework when it's easy (and fun) to just write my own? Which will also work exactly as I think it should work.
Cheers!
# Shalabh
Python does not "appeal to programmers".
Every programmer I know did the same thing I did when they dipped their toe into python. "It imposes rules on whitespace??" And then they drag it to the recycle bin. A 30 second experience. Fixing THAT is what "Matters Most" for Python. Until you do, it will continue to be a niche.
# Bog
I almost did it myself (drag to recycle bin) when I first found Python. The only reason I kept it (and am I glad I did!) was because it was recommended by a coworker (programmer) whos skills I highly respected. Soon I realized there was really nothing to be afraid of, my productivity soared and now it's my first choice for most tasks. It was an important life lesson too - to be wary of my own prejudice when encountering something new. I now realize my traumatic experience with COBOL in school, and the fact that most languages don't use whitespace, had built my prejudice.
Anyway, Python went from 8% to 14% usage in one year (fastest growing language in the enterprise) according to this InfoWorld article - http://www.infoworld.com/article/04/09/24/39FErrdev_1.html?s=feature . I'd also suggest you look at the python.org jobs page and quotes page to see what people are doing with Python. That might make you rethink your 'niche' comment.
Cheers!
If the shape of your whitespace is your idea of self-expression, you can't really be that much of a "programmer".
# arimasp
My idea of "self-expression"? Huh, what am I, picasso? The shape of my whitespace is my idea of organization, that's what it is. (And so I guess writing organized code doesn't make me "much of a 'programmer'" according to your comment.)
# Bog
This thing about indentation/whitespace has dogged python forever. Personally, I think it is a great feature, but for those who won't touch python without curly braces, there is an easy fix: Just accept an alternate form of blocks, which is: xxx ... { code # Note: code here is deliberately randomly indented. code code } instead of xxx ... : code code code right in the Python compiler. (I can hear all you python zealots scream so loud :) since it breaks the 'one way to do it', and pushes a stake right through the heart of the indentation based readability, but hey why not? I have often thought of whipping out a perl (sorry) preprocessor script that could take such braces based syntax and turn that to standard indentation based syntax, but it is best done right within the python compiler and interpretor: maybe allow a new command-line option that allows for braces based blocks. /Nara
Real programmers don't drag programs to recycle bins. ;)
If Python were like "every other" programming language, what would be its advantage? The Monty Python tie-in?
Python does appeal to programmers.  Or that super programmer down the row who you respect, because he really is very good at programming keeps telling you Python is worth learning. You finally break down and start playing with it. Or you think LISP is really cool, but not really practical these days and you look to Python.
Whatever, but it takes some effort to learn a new language. If the new language were just like your old language, you wouldn't be making the effort to learn it.
Programmers impose rules on whitespace themselves. Would you like to read a program written in Perl (or whatever) where the programmer used no whitespace to denote structure? Or worse yet had incorrect indentation? It would be horrible. The first thing you'd do was fix it, curly braces or not. You need the whitespace. So who cares about the curly braces? They don't matter. Indentation is what matters.
I ran across a really good quote recently. It kind of fits the whole whitespace argument. The thought being that the way Python does whitespace is a really good idea:
"Don't worry about people stealing your ideas. If the ideas are any good you'll have to ram them down people's throats" - Howard Aiken, computer scientist.
That's odd. I had the opposite experience. Personally, it took a few minutes for me to get over the whitespace, but most of the programmers I've introduced it to haven't cared. Of course, many of them are used to various 4gl-ish vertical-app languages, so maybe it's not too representative...
My experience is that there are two kinds of programmers: Those that want the best tool for the job (and are willing to adapt a little to best leverage the tool) and those who trash anything not "just like" what they're already used to.
I use Python primarily for doing network management housekeeping on a university network (802.1x, mac-based radius auth, etc). Most of the comments I see about making changes to python make me nervous, as they seem to ignore any "non-web" usage in a rush to "beat PHP" or whatever.
# Robert
As a Python lover who also uses PHP extensively, I find that Spyce ( http://spyce.sourceforge.net ) is the Python web programming solution that is closest in spirit to PHP. With Spyce, all the right ingredients that made for PHP's [initial] success, such as immediacy of learning and integration into an HTML page, are made available while straying as little as possible from how you normally do Python programming (read: nothing much new to learn).
Spyce's Pythonic design involves a small amount of carefully designed features which:
- provides powerful HTML-integration features more on par with JSP (such as rockin' custom tags) than PHP (whose tag-related features you need to boost with outside help such as Smarty)
- retains the ease and cleanliness of basic PHP
- lets you reuse what Python already does well, with the least distraction
- allows extension with template processors (such as Cheetah)
Spyce has been around for a while and mature and stable enough for 'heavy lifting'. It was what I settled on after evaluating roughly a dozen other Python web frameworks including CherryPy, Quixote, Skunk, Zope, and Webware, none of which, imo, offered the clarity and ease of HTML integration that PHP provides and thus were not satisfactory alternatives.
Spyce can be made to run on commodity hosting that supports CGI and Python (99.9% do) easily by:
- just copying over the Spyce files under a cgi-bin directory
- creating a 2-3 line .htaccess file in a directory where your .spy files will run
Voila! You now have Spyce working together with all the features that a Python program on that server has access to (including MySQL, SMTP, etc...)
Spyce can also work on top of fastCGI, mod_python as well as its own server proxying behind Apache or other web server.
Oh... and before I forget... if its default [[ and ]] delimiter tags turn you off, you can use the more conventional <% and %> instead.
The final icing on the cake is that the free SciTE ( http://www.scintilla.org ) editor can beautifully highlight HTML, Javascript and Spyce code separately when you use <% %> and enable 'asp.default.language=3' in its 'html.properties' configuration file!
Well, I agree - spyce is fine. Howerver, since I have found CherryPy...No. Spyce is -too- close to php programming for me. I think that the way cp2 is going is better for web programming. However, there is one thing cp needs - mod_cherrypy :-]
I used Spyce, but now mod_python supports inline code, so for med Spyce became redundant.
Hi Ian. I agree with you absolutely. Too bad Guido do not concern web development that much. Even if he does not do web related work directly he should deputize a team to come up with a solution within certain time frame. If ever Python come out with any clout like Rudy on rail I can see its user base easily double or triple.
Talking about Ruby on rail its momentum does startle me. But I think more importantly it should be consider an inspiration for the Python community. It is easy for a talented Python developer to crank out a well designed framework. But what really counts is the user base and public awareness. The way Ruby on rail takes on the audience shows there are still opportunities in web development, and that PHP and Java still sucks.
It might be insane but, why Guido should be concern with every framework built atop Python? As far as I see it, it is not really his business after all.
But, on the other hand, I totally agree with Jonathan when he says that the Python community should only focus on some solutions though I think we should act on two levels.
- One solution that is very simple and very pythonic so that it attracts newbies. PHP works great because it doesn't force you to use a complex framework like J2EE or .NET
- Another solution for bigger structures like J2EE and .NET offer. There Zope and Twisted might be welcome (I don't like them to be fair but that's personnal taste.)
I've found all that burden around Rail really sane for the Python community which sometimes is really quick to think Python is the best so we don't need to keep on going on. Python community is a bit self-centric sometimes where Ruby community is not ('till now).
That's why I also don't think Guido should have anything to decide in terms of web frameworks. We can't rely only on one unique individual.
Just y 2 cents...
# S.
Why should Guido concern about web framework? Let's scroll back to the top to check on Ian's key premise.
"Resolving Python's problems with web programming is the most important thing we can do to market Python."
So Guido does not directly invoked in any web development himself. But I bet he would concern about the market of Python and he would be a cheif evangelist himself. And here our believe is positioning Python as a premier web development tool is the top most important thing you can do to market Python.
I don't expect Guido to have any role in designing of picking the best framework. The best thing he can do is to be a catalyst to help the community to arrive to a better solution.
By the way I like your idea of having several layers of framework (like JSP and Struts). If the higher layer is build on top of lower layer that would be even better.
Couldn't agree more. Have developed untold lines of software in everything from Fortran/Basic/Pascal/Forth/APL, to C/C++/ObjC/Java and Python. While I still use C/C++/ObjC/Java occasionally, I stick to Python whenever I can because it is just better. That said however, I share the concern for Python's future because I'm selfish and don't want my Python relegated to the scrape heap of unsupported languages somewhere down the road. I fear that Python's inability to cohesively embrace the challenge of web-based apps provides too much opening for some lesser species to proliferate and push out other options in the same way the the Unix community failed to create a cohesive offering and allowed the M$ virus to proliferate so widely. I don't mind having lots of choices (I've tried most of the web app development frameworks out there at one time or another), but I would like to see a really well-engineered choice integrated into the Standard Library so that the community as a whole could focus its resources and deliver a knock-out punch in the world of web apps. Add my vote should you present this to the Powers That Be ...# Douglas Beethe
I do agree on what Ian is saying, but I want to go forward. I've tried to write down the requirements the 'ultimate' Python web framework needs to comply to. PHP supports all of these:
- a template syntax that can mix fixed HTML with generated code. Sadly enough the whitespace of Python does not fit very well with HTML. There have been numerous attemps to overcome this (Quixote ptl, psp, ...) but none of them are as easy as PHP. I think this is the main reason why Python keeps on struggling to become a very popular Web language. This one will be the main hurdle to get an agreement on because of the legacy of the existing templates systems
- a simple relationship: a web page is a file. Storing web pages in an object database (the Zope way) only increases complexity: it requires specific tools to do ftp, webdav, ..., while it does not bring anything more than the file system does. A web page can be an object if we stick to the 'one object in one file' rule.
- a RDMBS back end. While pyhton lovers will argue that a ptyhon object store is more pythonic, these object stores lacks the features of RDMBS like MySQL and Oracle: robustness, backup/restore , scalability, replication, transactional behaviour, ... People are scared to put 2 Gb of critical business data in an 'unknown' OO database. SQLObject might do the job to link to the back-end.
- non stop behaviour. Most Python web framework requires you to stop and restart the framework when you add/edit/delete a web page. In most case a web page is directly linked to a Python object, which cannot be easily removed/modified from a running Python process. This behaviour is not acceptable for operational sites. Or we need to go away from the paradigm 'a web page is a Python object' or we need to solve remove/modify objects issue.
- a decent release policy (and not like Webware who lives in the SVN repository for the last 2 years)
- a good Apache integration (both release 1 and 2)
- a good persistent session management (not like Quixote). PHP dumps session info in ordinary files, which is not nice, but it works.
Looking at the existing web framework we have (Quixote, Webware, CherryPy, SnakeLets, mod_python, ...) all of the frameworks miss at least one. Only if we have a framework that complies to all of these requirements, we can go for the last one:
- a solution that integrates nicely the different components, having a single integrated configuration file, installation script, start/stop script and documentation
OMFG. The amount of hand-wringing going on over this topic is beyond belief.
Is the sky going to fall if Python doesn't come up with a draconian "web standard"?
Are you all so eager to have legions of script kiddies using Python, leaving you with huge messes to clean up?
I do nothing but web development for businesses, and I use Python, and all this talk about how hard it is to get started with framework X, Y, or Z is utter balderdash.
Sorry if I sound harsh, but you guys are spouting nonsense, and encouraging some "sense of urgency" among each other in the same way that mass hysteria impacts mob behavior.
LDB
# L. Daniel Burr
I agree although a little more moderately. I am a Python web developer and have been able to meet any customer requirements using either Webware or Plone/Zope.
And I would argue that the reality is Zope has become THE Python Web framework. Does that mean we should stop working on WSGI? No. But Zope and Plone now have significant backing from large organizations. Plone/Zope is the official web platform for several federal agencies including NASA, NOAA, and the U.S. Navy. And even more large commercial companies have settled on Zope including Intuit and SGI. Like it or not that's a lot of market share and name recognition. Plus there are plenty of mom and pop sites running Zope as well. It seems that the Web has declared Plone/Zope a definite leader among web frameworks in general.
And as far as the core language, Python's weak XML support is a far greater threat to its success as both a web and general purpose language.
Web services and other web-related XML standards are becoming more important than any given web framework or paradigm in software development. J2EE and .NET are killing Python in the SOAP arena not to mention all of the other places where XML is applied heavily.
Count how many SOAP examples are in the latest edition of the "Python Cookbook" (Answer: 0). Also count the number of core Python libraries related to Web/Network programming versus the ones related to modern XML standards.
And if you still have doubts about Python's suitability and scalability for Web applications just ask Google how they feel about it :-)
# Joel
Your more moderate agreement is probably more productive that my rant, I readily concede ;)
As for your point about Zope being the de-facto standard, I agree. I'd also point out that there has been plenty of conversation about finding ways to use the Twisted framework for Zope's network plumbing, so that Zope can be more focused on its core problem domains. Also, Jim Fulton (the Zope Pope) has stated that he would like to incorporate some of the desirable features of the Nevow framework. The Twisted folks, for their part, have dropped their Interface implementation in favor of the one used in Zope 3.
SQLObject is a de-facto standard these days, too.
Now, my point in rambling on about this stuff is that I like the standardization that I've described above, because it happened naturally. Nobody had a big summit to lay down the law or make sweeping pronouncements. I hate the kind of standardization being argued for in this topic, because it is an attempt to dictate a "winner" technology stack, instead of letting the winning combination be determined by the only measure that matters, which is utilization in the field.
The irony here is that there is so much talk about creating a standard, about limiting choices, about making it easy for developers to avoid thinking about which technology to use when building a web application; Rails is just a framework that somebody built to solve some problems, and it happens to be evolving into a de-facto standard in Ruby-land. No voting or consensus involved, just natural forces at work.
What is being proposed here will fail, because it is an artificial, design-by-committee solution to a fictitious problem. Somebody, anybody, please give me a real example of how it is so easy to do X in Rails/PHP/whatever, and how it is just so much harder to do in Zope/Twisted/whatever.
LDB
While on some level "standards" are decided after the fact in the Open Source community, but they are rooted in conscious decisions. No one said that Jim Fulton had special authority over Nevow, Python web programming, or anything -- the informal authority he has was gained through his action and involvement, and leveraging the authority from a popular framework (Zope). And even in the presence of Zope Corp, he still doesn't get a pass when it comes to authority in that community. "Authority" in all these cases still only means that people choose to listen to him, not that he can actually make people do much (at least outside of Zope Corp).
In turn he chooses things consciously and strategically. Many people in the community do this -- they choose something, either to follow the opinions of others, to start out on their own, to become part of some project -- based on conscious decisions that include their imagination -- what they think will happen with projects, how they personally feel invested or alienated from a project, how they imagine a design fitting into their unknown future developments. Not all programmers do this -- but almost all the ones that matter do. Developer-users -- people who just download packages and use them in their isolated environments -- have very little effect on the code or community. Imaginative programmers are what make things happen, and that's who I'm trying to appeal to with most of what I write on this blog, because sometimes code alone isn't enough to spark someone's imagination. And sometimes I don't have code to hawk anyway ;)
In conclusion, because I don't think I've made my point entirely clear yet: on one level you can look at the community and its decisions as an organic process of evolution. And seeing that you can think that conscious decisions and strategies don't matter, that standards can't mean much, that the best man will always win in time. But the other side is that the community is made up entirely of individuals -- it seems organic because there are no authoritative institutions, no authoritative decisions outside of what an individual chooses for him or herself. But individuals make decisions for very conscious and personal reasons, and though it's a different process to appeal to a community of many individuals, it's still worth doing.
"Are you all so eager to have legions of script kiddies using Python, leaving you with huge messes to clean up?"
That statement represents part of the problem: elitist, short-sighted, unproductive, hand-waving.
Would you prefer cleaning up huge messes written with messy tools?
A regressive attitude will not make the problem go away.
# anonymous
I'm not the one hand-waving here, my anonymous friend. Hand-waving is an attempt to gloss over the difficulties of solving a particular problem.
My whole point is that I don't think there is any problem at all, much less some dire problem that threatens the future of all Pythonistas everywhere.
You can engage in further ad-hominem attacks, but that is hardly productive either, and you seem to care about being productive.
LDB
"Hand-waving is an attempt to gloss over the difficulties of solving a particular problem."
"My whole point is that I don't think there is any problem at all"
Exactly. You don't think, others do.
"You can engage in further ad-hominem attacks, but that is hardly productive either, and you seem to care about being productive."
Your reply lacks any productive statements.
"The amount of hand-wringing going on over this topic is beyond belief."
"Are you all so eager to have legions of script kiddies using Python, leaving you with huge messes to clean up?"
 While ignoring the substance of my response? And the numerous posts in agreement with the original blog entry? And those mapping out positive goals and directions? In other words; people like you are part of the problem. Solutions will proceed with or without you.
# anonymous
It would be impossible to for me to ignore the substance of your response, Anonymous Coward, there being no substance for me to ignore.
You have supported your contention that there is some dire problem here by basically saying "See? Lots of people here agree with me. I must be right!" Hardly conclusive evidence.
On the other hand, I have made it clear that I am speaking from my own personal experience, right, wrong, or otherwise. If you consider yourself to be more credible, try signing your name for a change, and speaking to your own experiences. That would be productive.
As for solutions moving forward with or without people like myself, you are being a bit unrealistic. Preaching to the choir of like-minded people who believe that "something must be done or Ruby/PHP will beat us!" isn't going to solve the problem you claim exists. If anything, you have to sell the sceptics and nay-sayers first.
So convince me. I've already stated my reasons for believing as I do. You haven't done anything except attack my right to do so.
LDB
"You haven't done anything except attack my right to do so."
You are projecting again.
# anonymous
Your failure to perceive the problem is remarkable but altogether unhelpful.
# s
I have to agree. I have been through 3 generations of web application development. Circa 1996 1st used Delphi to do CGI programming on O'Reilly Website webserver (!). 2nd Circa 2000 Programmed Java servlets on Sun Javawebserver running on Windows NT interacting with with Informix on HP-UX . 3rd 2003 to now PHP on linux/apache with Postgresql. Staying there until something better...
PHP is pretty ragged sometimes but I have found that people can become productive very quickly, there are lots of good examples (or at least examples). I am leery of the move to "all objects" php5. I have never seen anything that was just impossible in PHP, and usually there is a pre-existing built in function to use, or a PEAR class that can be installed.
I like python a lot and have written extensively in it, doing large Postgresql database load operations in large scale conversions from legacy systems. At one point I was convinced that it would be worth it to do web programming in python but everything just seems to be so immature, and the python web developer base is tiny.
Zope seems ridiculously complex, though it seems to be the poster child for "big time" python success. And it is slow, admit it. And given there are probably dozens of good CMS's in PHP (mediawiki, mambo, drupal, etc) why not skip it. Just because it's "famous in Europe?" (like Slim Whitman?). I think Zope has enough momentum to actually make a buck as a consultant if you get Zope expertise but that is just a guess.
Twisted Matrix seems interesting but practical examples/tutorials of significant production non-toy web apps (e-commerce, CMS, business system/database apps etc.) seem missing. "It does everything and does it fast", "Write an http server in 2 lines!" etc sounds great but where are the real life examples?
I agree with earlier comment that requiring Apache 2 to run latest mod_python is a drag for PHP users. I tried it but compared to ease of development in apache anything and PHP why bother.
I would not worry that much about Rails. Like the twisted matrix crowd there is the tone of "look everything is easy!, a site built in 100 lines of code" and again, where are the real life, in use applications that I can download and look at and which don't look like a high school CS term project.
Anyhow, IMYO, too late for python to make a significant impact for web development, the python news always seems to be dominated by arcane and obscure discussions (i.e decorators). IronPython may be the next big deal, but the alliance with the dark forces of Microsoft and .Net will alienate many.
My two cents...
# Rick Burque
I also totaly agree. I ended up doing a big web project in php because it was too hard to do in zope (though we tried for 3 months) and mod_python was to difficult too use compared to php. I really wish that I had done it in python now, but, its too late for that. Now I have this giant beast, albeit written well, its written in a crappy language. If python were convenient to use on the web it would be a power house. I don't want to start flaming, but after using php for a few years I can confidently say its a mess, a real scritping language. I could almost write better scripts in bash ;) --not really. I'm starting a new project and looking at zope3, but I really wish python was as clean an easy as php, and had a tool-kit as smart as rails. Alas there is still hope! Let us unite and crush the competition, lest we be the beta-max of the web. Superior, but eventualy useless.
A quick search for "webware" in the comments came up with nothing. I'm a web developer with a background in cold fusion. I've been using python to code background tasks like importing and exporting data and just about anything I can think of. I recently I started playing with webware because I want to see what python can do on the web. What is everyone's comments on webware? Is it worth it's weight?
From the looks of it, a mixture of SQLObjects, Cheetah and Webware would allow you to build a damn good/flexible webapp in a couple hours.
For those who don't know of webware, http://www.webwareforpython.org/ , it's an application server in python that resembles concepts in J2EE. Servlets, PSP, all that jazz. It's open source too. Gotta love that. Thanks.
# Eric Moritz
i'm having success with webware, but it too is missing what was identified elsewhere in this article (wrt python web frameworks in general): complete, coherent documentation and start-to-finish examples that clearly demonstrate all the major elements of webware/cheetah/etc working together to create a typical web app.# mike
also forgot to mention that webware's release schedule is agonizing. some people aren't comfortable living out of version control for their production code.
I personally see WSGIKit as the future of Webware, and it provides a cleaned-up Webware experience right now. I like Webware, but it has some issues related to testability and coupling; WSGIKit started out as an effort to resolve those problems (though I think it has become more than that).
PHP is pretty ragged sometimes but I have found that people can become productive very quickly, there are lots of good examples (or at least examples). I am leery of the move to "all objects" php5. I have never seen anything that was just impossible in PHP, and usually there is a pre-existing built in function to use, or a PEAR class that can be installed.
Ian, I completely agree with your analysis. I am not a developper, rather a 'power user'. My first goal is to get things done quickly and cleanly. I often use python for different purposes and I have tried (at the tutorial level) most of available python web frameworks.
I think we need a kind of easyPHP all-in-one package (including python and a db) with a simple demo application that you can immediately fire and which acts as the first step by step tutorial.
I am not enough skilled to launch a new collaborative project, but I will love to participate to one like this if some python expert would accept to start and pilot the project. I think python community should try to leverage the huge base of good-but-not-experts programmers available among its members to participate to projects like this. This probably leads to invent a new kind of collaboration, simpler to start with for newcomers. How to work with 20-30 good programmers who can provide a few hours of work per week, instead of 4-5 experts much more involved.
If Python community would manage to invent this new collaborative method, involving a lot of good pragmatic developers (uninterested in overdesigning), I am convinced that the new Python-for-web package would not need more than a few months to emerge.
Am I a poor lonesome-good-enough-but-not-expert python programmer ?
If not, write down a line on this blog, and perhaps some Python guru/leader will think it's a pity not to use such a treasure of pragmatic and available task-force.
# Jean-Pierre Camus
I think we need a kind of easyPHP all-in-one package (including python and a db) with a simple demo application that you can immediately fire and which acts as the first step by step tutorial.
That sounds a lot like Karrigell.
Simplicity is the key. Make testing changes as quick as possible.
So far my python web development has been done with the cgi module, zope, plone, twisted, simplehttpserver, mod_python, and now lighttpd + wsgi. So like a lot of people I have tried all sorts of things. I also use php, and a little java on some projects(not ones I have started). Here is my brain dump of where I am at, and what I'm doing with python web dev stuff.
So far I think plone+zope is the most advanced, and popular of all the python web development frameworks at the moment. It is really simple to install and get a site up. It is simple in some respects, if all you want is in the default plone site. However making modifications to it requires a lot of learning. It has sooo many users at the moment compared to other frameworks, that it will eventually have enough modifications available to do many common web tasks. Also getting good performance out of plone is hard.
I like the subway effort. That's a really good idea, and I wish them luck. Cheetah + SQLObject + WSCGI are pretty good building blocks, and are some of the things I'm using on my latest python web project. I want to see how they do automatic reloading of templates.
Simple. Five lines or less to do hello world. Simple. Twenty lines or less to do a CRUD application. Scalable. Should be able to be moved to run on multiple servers easily. Default config should handle lots of users.
Can any python framework get 'hello world' done in five lines or less of explanation? What about a CRUD app in less than 20 lines of explanation?
Archetypes is probably the only one I can think of that can do a CRUD one quickly, and simply.
Here is a less than five line php hello world explanation. It will work on most webservers around, so no need to describe how to install php. Not describing how to install php makes the explanation shorter than a python one would be.
- open a new file in notepad/vi.
- write into it this text: <?php echo 'hello world'; php?>
- save file as index.php
- upload to your webserver.
OR for unix shell users: echo "<? echo 'hello world'; ?>" > index.php
That is simple.
Doing a CRUD application with the basic php libraries is quite hard. There are many more frameworks in php than there is in python.
I really like WSGI, and I will be using that for my new stuff. With wide spread WSCGI support there should be able to run those types of applications more easily without having to do complicated install instructions. It should make using these various python frameworks easier. Luckily I don't have too much python code to convert on my latest project.
My latest project is based on these things(and others).
- Cheetah for templates.
- can be made to work nicely with html editors. nvu, and dreamweaver with the added <!--# stuff around cheetah stuff.
- I made a basic inheritence system for templates on the file system. It checks to see if there is a template for each content types insert, update, and list things. Otherwise it uses the default templates for these things. This is nice, as the designers do not need to do up nice looking interfaces for every object, and I can prototype things really quickly without having to make up html interfaces.
- SQLObject for SQL stuff.
- makes it easy to get an app up quickly. Less insert, update, delete, select to write. Can use sql directly later for performance/more complex queries.
- Simplexmlrpc server.
- for allowing integration into other peoples systems.
- WSGI
- for a common request object. So I should be able to move servers around if needed.
- lighttpd + fastcgi
- instead of apache for performance and security.
Using this howto for WSGI and lighttpd together. http://www.cleverdevil.org/computing/24/python-fastcgi-wsgi-and-lighttpd
I think lighttpd + fastcgi + python is an awesome combination. Each user can have their python processes run under their own userid. Then operating system security, and resource constraints can actually be used! You should get much better performance than apache + mod_php. You can have processes running on multiple machines with load balancing really easy to set up.
I tried using mod_python... but I prefer to have single, or separate processes running which makes caching data easier. Caching database connections, and other data is hard with multithreads/multi processes used in mod_python.
What is missing from this stuff I have jumbled together:
- Validation library.
writing lots of stuff manually so far. Taking bits from old projects, and random places on the net.
sqlobject protects from some sorts of errors with sql.
- I require js validation stuff for my forms as well. Because for some interfaces detecting errors early can save the users lots of time.
- This needs to sit outside of the templates, and be inserted based on the id, or class of the html elements.
something like mod_security which can allow me to quickly block, or warn me about certain requests based on regexes.
- Simplicity.
- So far I do not have simplicity. I can't even think how to write hello world off the top of my head.
- Not as data driven as I would like.
- I still have to define some things in code. Not good for different language access by other programs.
- SQLObject.
- I need to make sure my load balancing works correctly with SQLObject, as it is not designed to be multi process friendly(I think?).
- Need to figure out a good way to keep track of database schema changes, as I create my tables using SQLObject.
Python has been fun for web dev :) Certainly heaps easier than php. I can write a CRUD application in a few pages of code(basically an SQLObject with some extra meta data in there).
The full pipeline of development needs to be taken into account. Ie. design, testing and maintenance. The problem is this is different for most people. I mainly work with at max four other programmers, and a few designers. Which is quite different from one person doing the whole thing.
Simple. Simple for designers, programmers, and users to change things.
should be able to use the tools they like to make changes(dreamweaver, nvu).
- should be able to change templates at runtime. Ie reload a page at any point in a web application.
- whilst looking at the page in your web browser, click a button and have the template for a section of the page open up in your editor of choice. This can be done with javascript talking to your app server, and having it find out the template and tell your editor to open it.
saving a module, template, or db definition should automatically upgrade the development version. Once a file changes, the web app should reload it. This makes development lots quicker.
- users should be able to add reports easily themselves.
- one easy way to do this is to save searches. Plone makes this easy to do by copying the url of the search and saving it somewhere.
- making objects easily searchable is a good good for making reports simple to generate.
- outputting data in formats that users can make reports in easily is a good idea. ie in excel/csv format for spreadsheets.
- graphing software built into the web app, so that things can be graphed, and traced easily.
Security.
Not many web apps use database, or OS security to its fullest. Why should the appserver with a user logged in with anonymous rights have the same permissions to the database as an admin user logged into the app server. If you have different appservers running for different users(app, db, os users) then you can use those layers of security to protect yourself better. The root/god/admin user for you app may even be able to log into different computers than anonymous users. If you have the app server for anonymous users running with restricted OS, and database access then you can provide different resource limits. Eg limiting the memory of the user your anonymous appserver runs as, and limiting their CPU use, their number of open files, number of open sockets, firewall rules etc.
There's lots of words for you... I think python allready is quite good place for web development, and it is getting better all the time :)
# illume
FWIW, WSGIKit tries to achieve many of these things -- some directly, some indirectly. Setup is pretty quick and directed (not well tested outside of my Linux box, but that just takes time). It doesn't support a whole lot of servers at this point, but I just haven't put any effort into that yet -- the WSGI part should make this easy, but easy enough that it's not a Hard Problem so I feel find putting it off ;)
The first tutorial uses Webware APIs and ZPT. The ZPT library (ZPTKit) handles template reloading so that works fine. WSGIKit itself has a system for restarting the server when code is changed, so that handles other files being edited.
SQLObject is growing some management tools (sqlobject-admin) which will help with upgrading schemas and managing the upgrade of devel and live servers.
So it's getting there. It takes at least a little while, though ;) Mostly I need to bring more developers in.
This is how I write the "Hello World" program in Karrigell:
print 'Hello World!'
One line. Really!
# Luis
It's been said in earlier comments but I think it's important. If you're after mind-share you need an approach that will work on commodity shared hosting. Even mod_python is not widely installed by shared hosters.
The other thing that kills off any casual interest is the dependency of so much Python produced web-related code on use of a command line to run "python setup.py install". Shared hosting with command line access is very much the exception.
# richard shea
If we think that corporate use-growth is important, then:
- I think web-dev is key because it's hard to imagine many corporate developers starting new fat-client initiatives (outside of certain areas like trading desks).
- webservices and other approaches to integrate with legacy apps also become key
- cheap vanilla hosting support isn't much of an issue.
- SQL interaction is key; and allowing developers to think in sql/relational terms is an important leverage point
- flexible security management, possibly driven from a pre-existing system (LDAP? or maybe in-the-db rules?) is needed
- ...based on text files rather than storing objects in some sort of binary library is key to leverage existing management tools (CVS, etc.)
- some sort of CRUD generator might make sense for at least managing some simple reference tables, even if you don't keep that code for the main user stories...
interesting discussion.
i'm rather new to this web programming world, but i've spent the better part of the last 6 months developing a web application framework to build a commercial web app. a considerable amount of time was spent considering the existing open source solutions there were out there at the time (i hate to reinvent the wheel). (FWIW, i ended up using something very simple and nice called "draco" and plugging SQLobject into it, and then writing custom code on top of it.)
i think that there are no easy answers to the python web programming issue. there are too many different pieces and too many varying desires. it's a very exciting sphere. some people what an object db backend (better conceptual mapping to data structures), some people want SQL db backends (solid, fast, tried tested and true). some people want to use apache as a front-end server, some people want a single running process (e.g. Zope). some people like a distributed model of treatment (many childs running, e.g. mod_python), some people like an event-based model (simpler, e.g. twisted). some people want one thing to do it all (zope), some people want to provide independent pluggeable pieces (e.g. python webobjects). some like templates (zpt, etc.), some don't (e.g. using stan directly to generate a tree). some people like to write low-level unit tests (e.g. zope), some people like to test from the browser's point-of-view (e.g. using mechanize).
lots of options. you want to get started in a few hours?
i haven't seen anything serious out there that allows you to create a SERIOUS app in a few hours. sorry. it's either too complicated, or there is something you have to fiddle with to make it do what you want. i don't even believe it's possible to do so. and it doesn't even matter. you can't do this in a few hours if you're trying to solve a real complex problem, with lots of customers banging on your app at the same time (and you HAVE to simulate that). sure, you can write a cool little happy demo in ruby-on-rails (or whatever hello world comes with the python frameworks out there) but writing a seriously complex app will ALWAYS require of you that at some point you will be debugging some deep problem down to the bare metal, some race condition, some hard-to-find bug that'll suck an afternoon (or even sometimes a couple of days). software is hard. the more experience you acquire, the more you become aware of this. it just won't go in any other way than getting hurt year and year by the same tendencies to want go fast fast faster. writing a serious product REQUIRES of you that you write a lot of tests (or spend painful hours fixing bugs reported by customers). this will seem a bit lame, but IMHO in a few hours you barely have time to draw a couple of diagrams and explicit the most important issues in a document. if you have much development experience, you've been hurt in the right places and you can sort-of "foresee" where it's likely to hurt again. you walk about a little more carefully. if you think that you can finish a production-ready app in a few HOURS you're fooled. you've got NOTHING. VALUE in software, real value, is acquired with testing and with real usage from real customers, and years of maintenance, not once you finished the code. once you're code complete, you still have no value. you have a good intentions and a first step. this is why there is still a lot of COBOL code around. it's got great VALUE. it's _working_ code. it solves _real_ problems.
this reminds me so much of OLE/COM. you had these macros that were supposed to do all this magic CoCreate factory stuff for you, but as SOON as you needed to move a little bit away from what was intended, you had to write all this goo by hand. same with the web programming frameworks: do NOT judge it by its "hello world". this is why i'm not the least impressed with the ruby-on-rails demos i've seen. all i saw was that doing the trivial thing is easy, and it comes out of the box. can i really develop a large serious app with it? (Maybe yes, maybe no. I still don't know.)
so who cares if the newbies can't get in easily? who cares if the "hello world" is nicer with ruby-on-rails? who cares if there is nothing comparable in python? who cares if PHP is attracting a lot of people?
in the end, if you're a hacker, you'll put the pieces that you need together and you'll just make it work. you'll get down to the bare metal and just f**ing fix it. because you have to. put your hands in the grinder and the grease. i truly wish that we had found the magic formula to make software development easy for everyone but we're not there yet. you still gotta do your homework, learn your data structures and algorithms, and learn practical DIY skills.
and you know that even those writing PHP code today will have to do that once their app becomes anything close to serious. and they'll suffer. the grease pit of python is not bad at all if you ask me. the C code embedding is super nice, and the modules code is simple to understand, when you have to go there. anyway, they'll eventually gain experience. and they'll eventually look at it again and recognize the elegance of Python and that in the end, it's not that hard to pry all the pieces you need to make it work, and it's worth the boot-up effort. those who don't get it will keep suffering in their PHP world.
so how do we make Python more accessible to the web programming world? the language already rocks. the libraries are all there. all you have to do is provide one good solution to get the newbies hooked up, and one good solution that makes it easy to install as PHP. AFAI can see all that is needed is easy deployment. what is the big deal? people with serious intentions will get over the lack of quickstart appeal.
cheers,
# Martin
"so who cares if the newbies can't get in easily?"
Anyone running a project on a tight budget! Yes, PHP is short-sighted but newbies can get the demoware ready quickly and cheaply with PHP. When choosing two from [good, fast, cheap] tight budgets imply cheap. Fast will make typical management happier than late goods.
Not caring is always an option, quality survives. Lisp, Smalltalk, etc. Those tools also generate these debates.
# anonymous
Could it be that Digital Creations's Zope is the reason why php has become more popular than python for web programming? Discuss.# anonymous
Oh no, not another!
I DISagree with you, and most of the people that have made comments here. I do not think that throwing vast amounts of resources on improving Python Web programming is going to do much more than make it adequate compared to the vast competition out there.
I think we could carve out a better niche for Python by making it the preferred language for scripting applications. When software suppliers don't just do the equivalent of packaging up their software for COM, but are actually delivering functionality by extending their core application using Python. For this to happen we need to focus more on things like KDE/OpenOffice/Gnome/Mozilla/... support.
I am not advocating abandoning WEB tools development in Python, I'm just observing that there are a lot of competitors there and whilst it is very important to show that Python can work there too, I don't think we can ever become the preferred language for web programming.
"I think we could carve out a better niche for Python by making it the preferred language for scripting applications."
I agree that Python applications are a valuable space, even as stand alone applications lose share to web applications.
If you meant embedding Python in applications then look to LUA's successes.
If you meant applications then look at the cross platform problems with wxWindows (Microsoft-specific), Qt (licensing issues), GTK (unMicrosoft-specific), and so on. Similar issues didn't stop Java (AWT's, Swing, and the unSun solutions). Or maybe they did, Java applications are novelties, OpenOffice and Eclipse are blimps.
My personal hope is that Parrot will be uncomplicated, fast, scalable, and very friendly to Python, even on Apache. Python on Parrot may attract those with the skills and mindset necessary to implement the desired Python modules.
# anonymous
I think what may be meant by "scripting applications", especially in the context of the excellent work going on with things like PyQt and PyKDE (and I imagine that PyGtk is pretty interesting too), is actually writing KDE and GNOME applications from scratch in Python instead of getting overexcited by some supposed "need for speed" that C and C++ offer and then finding out what kind of wheel reinvention has to go on to support things which Python already does very well.
But I have to take issue with the Qt licence jibe - the GPL should surely be good enough for open source applications, and if you're in the business of writing closed source commercial software for sale, but you can't make the money back to pay the licensing fees for Qt, perhaps you ought to consider another business model.
(As for Parrot, if it has finally moved off defining bizarre Perl 6 semantics as virtual machine instructions, perhaps it has a chance after all. Otherwise, I'm not holding out much hope for it, I'm afraid.)
# Paul Boddie
The posts in this thread indicate that people understand the issues involved, what is required to address the issues, and some are moving forward as I type this post. Instead of helping I will continue typing...
I should mention that these are all personal choices, for personal reasons, for personal projects. I'm not aware of any available Python GUI wages.
With that in mind: I choose to avoid writing KDE or GNOME specific applications. I don't keep up, I'm tired of it, I don't care if D-BUS wins or loses. Like Debian, GNOME seems to need more bizarre life and less dusty dust. KDE needs more speed. I prefer the GPL but will put up with the Python license. I suspect OpenOffice is a dead end but would be happy to be wrong. If the first post had mentioned Firefox instead of Mozilla I may have been more enthusiastic. Is it easier to embed Python or Lua? I don't know but would guess Lua wins (given its footprint and popularity). Extending applications externally begins to look like web services, which begins to look like the original subject. :)
"if you're in the business of writing closed source commercial software for sale, but you can't make the money back to pay the licensing fees for Qt, perhaps you ought to consider another business model"
Sure, the GPL creates similar conflicts for profit motives. I wrote
"If you meant applications then look at the cross platform problems with wxWindows (Microsoft-specific), Qt (licensing issues), GTK (unMicrosoft-specific), and so on."
I was thinking of the market confusion caused by TrollTech's licenses, and the various stances and rhetoric spewed over the years. Not my confusion, the markets. Qt license threads are not new.
 But my personal choices weren't the point of the quoted statement. The point was that there is not a substantial Python answer to the GUI question when the multiplatform planet is asking (commercial, anti-fascist, and all the rest). The planet sized version of the GUI question begins to look like the web version of the original subject too, and other languages have better answers. My hope is that a personal project might wag the long, killer app tail and that perhaps others will react by cleaning up the messes. If not, at least I am enjoying PyGTK (except for the Menu parent/tearable/popup breeding headaches).
While not caring about a solution for others, including the newbies is certainly a valid choice that choice also surrenders the benefit of network effects, the fun of being able to say "Python is a great language, fast, easy, productive, and goes 'ping'" and not have to conditionalize with "as long as you are smarter than me, have time to learn trivia, and can afford it".
As for Parrot...
I first became nervous when Ruby brought up Visual Basic compatibility. Then Dan disappeared. Maybe Parrot is a dead end. Sadly I think the Python VM is too. So something needs to come along, if not Parrot. Not that I care all that much; I hold my personal language choices close but am prepared to release them at any moment.
Under wages PHP is much easier to pitch. :)
# anonymous
<quote>If you meant applications then look at the cross platform problems with wxWindows (Microsoft-specific), </quote>
wxWindows is MS-specific, but wxGTK works fine for me under Linux. Either way, wx has generally been my GUI of choice. And, after reading this thread, I'm going to re-think my stance on php. The pages I have created with it are a mess.
BTW, as far as the hosters go, I'd say there are basically 4 levels of python support: 1. python's installed and you can run CGI with it 2. mod_python 3. zope 4. plone
I really should start evangelizing the joys of plone to try to convince more hosts to offer it. Then again, I'm not sure how hard it is to actually keep it running. Having it on my development box at home isn't really the same.
I think we could carve out a better niche for Python by making it the preferred language for scripting applications.
I disagree with you vigorously. Do we want to tread the steps of such language titans as Tcl and Lua? Ha! Scripting doesn't build community and doesn't build participation, and that matters tremendously to an open source project. Scripting isn't bad for the language, it's just not very good. Honestly, I've never usefully scripted any application with Python, and I've only uselessly done it once or twice.
Also, the idea we can dominate that position is silly. Anyone can create a language powerful enough for "scripting". And they do it all the time -- Lua, rep, JavaShell, yada yada. It's stiff competition for a fairly pointless niche.
If you mean gluing together applications, yeah, okay... but that's not even a niche, it's just an idea. And the web has as much to do with that as anything.
# Ian Bicking
To succeed in the web framework, Python will need to answer the following challenges:
To be ubiquitious, so that budding developers can find cheap hosting. There has to be sufficient low barriers to entry for the developers as well as the web hoster.
This requires
A supported python web framework distribution that can be easily deployed on the hosts. BSD and Linux versions required, with mysql client-libraries bundled. Like it or not, MySQL is still the dominant RBDMS on web hosts.
The web framework must also be easily deployed on Windows hosts, where most incidental developers start from. I believe the initial success of Zope is partly attributed to its availability on Windows.
Sane configuration and deployment story, so that a developer can deploy their tests to the web host consistently, even with different web hosters. This will decrease support costs for hosting companies, and please the developers no end.
(I'd include the configuration of the SQL connection in part of the deployment files).
My vision is that all web hosters which host this framework will automatically provide a user configuration file for this python process that includes mysql connection details automatically. The framework could also do the dirty work of synching between local and remote servers.
There is much we can learn from how Wordpress makes the installation painless.
Changes in python required
- Hosters must be able to effectively share a persistent python process between users.
- Modules must be 'safe' and module level variables must be cleaned between invocations. (How does Ruby on Rails hosting cope with this problem?)
- User modules must be unloaded after requests to conserve hosters resources.(Note: this will also solve the problem of module reloading in Python).
Provide a few prebuilt solutions (optional) and pretty templates (important) for solutions to be quickly implemented.
People learn by modifying existing applications. Most people though just want to deploy existing applications.
Can we have the equivalent of PHPBB on Python, or the PhpGalleries, or PhpShop?
I remember touching on this issue on Web-Sig last year. Although I recall that you had some sympathy for my position, others were far more interested in either trying to emulate Java, or favouring framework developers over web application developers (mainly because most people on the list were framework developers). I still think WSGI solves the wrong problem, and that its existence distracts from the fact that Python could, and should, be a great alternative to PHP or ASP, if the framework developers were willing to work together to create something approaching a standard, rather than offering a bewildering array of complex partial solutions.
Unfortunately this is just one of several problems holding Python back. A similar situation applies to multimedia, which Python doesn't support very well out of the box. I find it bewildering that we have lexical analysis, scheduling, and curses support as standard yet OpenGL support has to be downloaded along with its dependencies. Versioning issues (at least on Win32) are a significant problem for me too - there are all these new features in Python 2.4, yet I can't use it until the libraries I rely upon are recompiled/repackaged for this version, which may never happen. It is unfortunate that Python, despite being a language that is incredibly convenient to use, does not have the necessary structures in place to make its adoption equally convenient.
# Ben Sizer
I put this down to lack of respect for the developer and the customer. No one running a substantial application has the desire or incentive to rebuild every component that they depend on when upgrading from one version to another. With legacy apps, the source may be lost. There are technical solutions available through XPCOM or COM to avoid breaking binaries through versions.
I am starting to create a myspace type of application. I have two choices. Python and PHP. I am inclined to Python because of its cleaner code and strong oo, which I feel will be a huge asset in interlinking all the different types of content. Is python and cherry py an appropriate development platform, do I need anything else? My web host says that they support python (http://www.bluehost.com) but I will have to set up everything (like cherry py) myself on the server. Will it be difficult to get this running behind apache on blue host. What do you think are the relevant issues?
You say that programmers like Python and thus it deserves to do well in Web Programming space (or should do better anyway)....
But programmers also apparently like Ruby quite a bit and it IS doing well in the Web Programming space. So why does it really matter which of the Object Oriented, dynamic languages "makes it" in the web programming space? Maybe Ruby is 'making it' there for some good reasons. You often hear how similar the languages are, but really the mindset does seem to be different between them. I suspect that Ruby has something going for it that Python finds more difficult.
Anyway, I repeat the question in another way: Why not cheer Ruby along since it has most of the things you feel are important (except for the whitespace-as-syntax which most of us can live without anyway)?
# os
As a PHP programmer who have tried to pick up Python:
The number one thing that makes PHP easy to use is it's documentation. The thing with good examples and user comments together with the actual api makes PHP great for productivity. I found it hard to use Python because the documentation wasn't organized as nicely as the PHP docs was.
# John Nilsson
- Why are we obsessed with Ruby on Rails?
Ruby on Rails has made such a stir in the Python community because of the hype and popularity of it. Those of us who feel that Python is a superior language are kind of jealous of the hype. Why not Python? Why not a Python based web framework? Waaaaaa!
I've never used RoR, so I don't know whether I would like doing web development in it. I suspect I would like some aspects of it and not others (based on seeing Dave Thomas's presentation in it at a conference I went to). I suspect after the hype starts dying down, we'll get to hear about RoR's drawbacks.
- What makes up a good web framework?
I agree that a really good Pythonic web framework would be neat. However different people have different ideas about what makes up a good web framework. And I'm not sure what a truly Pythonic framework would be like.
Personally I can't stand having code embedded with HTML. Yuck-o! Usually when I work on a project the HTML is done and redone by a graphic artisty person and I don't want code in there for them to muck up. I don't want to have to re-integrate code every time I get the HTML back. I want as little code as possible in my HTML please. That is a huge deal to me.
So things like Spyce and <insert your templating language of choice> are not very appealing to me. Heck, why not pure HTML with unique IDs on everything, and a parser that compiles the HTML into Python with a way to access all the ID'd HTML elements?
- What about ASP.NET?
I am a fan of ASP.NET actually. Yeah, boo hiss I know. Evil Microsoft, etc. I'm surprised someone hasn't developed something similar to ASP.NET in Python (look at Tapestry for Java and Prado for PHP). I don't think ASP.NET is perfect, but I think it has some really kick butt ideas that take the drudgery out of coding web forms and forms processing.
Server Controls are a great way to componentize HTML + logic. User Controls are a great way to re-use forms and all the form processing code. They aren't perfect, but they are pretty darned good.
- What about PHP?